Fault Injection Techniques and Tools

D ependability evaluation involves the study of failures and errors. The destructive nature of a crash and long error latency make it difficult to identify the causes of failures in the operational environment. It is particularly hard to recreate a failure scenario for a large, complex system. To identify and understand potential failures, we use an experiment-based approach for studying the dependability of a system. Such an approach is applied not only during the conception and design phases, but also during the prototype and operational phases. To take an experiment-based approach, we must first understand a system's architecture, structure, and behavior. Specifically, we need to know its tolerance for faults and failures, including its built-in detection and recovery mechanisms, 3 and we need specific instruments and tools to inject faults, create failures or errors, and monitor their effects. Engineers most often use low-cost, simulation-based fault injection to evaluate the dependability of a system that is in the conceptual and design phases. At this point, the system under study is only a series of high-level abstractions; implementation details have yet to be determined. Thus the system is simulated on the basis of simplified assumptions. Simulation-based fault injection, which assumes that errors or failures occur according to predetermined distribution, is useful for evaluating the effectiveness of fault-tolerant mechanisms and a system's dependability; it does provide timely feedback to system engineers. However, it requires accurate input parameters, which are difficult to supply: Design and technology changes often complicate the use of past measurements. Testing a prototype, on the other hand, allows us to evaluate the system without any assumptions about system design, which yields more accurate results. In prototype-based fault injection, we inject faults into the system to • identify dependability bottlenecks, • study system behavior in the presence of faults, • determine the coverage of error detection and recovery mechanisms, and • evaluate the effectiveness of fault tolerance mechanisms (such as reconfiguration schemes) and performance loss. To do prototype-based fault injection, faults are injected either at the hardware level (logical or electrical faults) or at the software level (code or data corruption) and the effects are monitored. The system used for evaluation can be either a prototype or a fully operational system. Injecting faults into an operational system can provide information about the failure process. However, fault injection is suitable for studying emulated faults only. It also fails to provide dependability measures …