Efficacy of AETG∗Test Cases as measured by code coverage

Software testing must be performed as efficiently as possible as it can be very expensive. Efficient testing requires a small set of tests that does a good job of locating program faults. The AETG system takes information about the program being tested and generates a small efficient set of test cases [1] [3] [4]. The AETG system has been used in several applications. These include testing communications protocols, telephone switch features, and screen validation software. While the AETG system produced small test sets for these applications, it was not possible in a field environment to do controlled experiments to measure the effectiveness of AETG tests sets. This report presents experiments measuring the effectiveness of AETG test sets. One measure of a test set’s effectiveness is code coverage. Some recent work indicates that good coverage is correlated with ability to find faults [5] [6]. We did several experiments measuring the code coverage of AETG test cases. The AETG test cases gave good coverage. These experiments also gave insight into modeling a system as AETG input. While this report focuses on code coverage, code coverage alone is not a sufficient gauge of a test set’s effectiveness. Tests run on a program missing required functionality may cover all the existing code without detecting that code is missing. Coverage of the program’s requirements is also needed. We did experiments using AETG to do requirements-based testing of screen validation code from two releases of a Bellcore system. By methodically testing the code against the requirements documents, the AETG system found faults not found during normal testing. Section 2 of this report reviews the AETG system. Section 3 provides information about code coverage measurements. Section 4 presents different test generation strategies. Section 5