Evolutionary Safety Testing of Embedded Control Software by Automatically Generating Compact Test Data Sequences

Whereas the verification of non-safety-related, embedded software typically focuses on demonstrating that the implementation fulfills its functional requirements, this is not sufficient for safety-relevant systems. In this case, the control software must also meet application-specific safety requirements. Safety requirements typically arise from the application of hazard and/or safety analysis techniques, e.g. FMEA, FTA or SHARD. During the downstream development process it must be shown that these requirements cannot be violated. This can be achieved utilizing different techniques. One way of providing evidence that violations of the safety properties identified cannot occur is to thoroughly test each of the safety requirements. This paper introduces Evolutionary Safety Testing (EST), a fully automated procedure for the safety testing of embedded control software. EST employs extended evolutionary algorithms in an optimization process which aggressively tries to find test data sequences that cause the test object to violate a given safety requirement. A compact description formalism for input sequences for safety testing is presented, which is compatible with description techniques used during other test process stages. This compact description allows 1) an efficient application of evolutionary algorithms (and other optimization techniques) and 2) the description of long test sequences necessary for the adequate stimulation of real-world systems. The objective function is designed in such a way that optimal values represent test data sequences which violate a given safety requirement. By means of repeated input sequence generation, software execution and the subsequent evaluation of the objective function each safety requirement is extensively tested. The use of EST for the safety testing of automotive control software is demonstrated using safety requirements of an adaptive cruise control (ACC) system. The EST approach can easily be integrated into an overall software test strategy which combines different test design techniques with specific test objectives.