Interactive Testing with HOL-TestGen

HOL-TestGen is a test environment for specification-based unit testing built upon the proof assistant Isabelle/HOL. While there is considerable skepticism with regard to interactive theorem provers in testing communities, we argue that they are a natural choice for (automated) symbolic computations underlying systematic tests. This holds in particular for the development of non-trivial formal test plans of complex software, where some parts of the overall activity inherently require guidance by a test engineer. We present the underlying methods for both Black-Box and White-Box testing in interactive unit test scenarios. As such, HOL-TestGen can also be understood as a unifying technical and conceptual framework for presenting and investigating the variety of unit test techniques in a logically consistent way. 1. Vision of HOL-TestGen Observation: • Any formal testcase-generation method is based on the solution of logical constraints. • Limits of constraint solvers hamper testcasegeneration techniques. Conclusion: =⇒ Testing should be integrated as automatic tactic in an interactive theorem proving environment =⇒ Testing and Verification may converge by using Explicit Test Hypothesis THYP’s =⇒ Correct Tool (cons. Embedding., derived rules) 2. How to use HOL-TestGen The abstract workflow of HOL-TestGen is divided into four phases: 1. the test specification, 2. generation of test cases (and a test theorem), 3. generation of test data (TD), and 4. the test execution (result verification) phase. Once a test theory is completed, documents can be generated that represent a formal test plan.