A Language for Test Case Refinement in the Test Template Framework

Model-based testing (MBT) generates test cases by analysing a formal model of the system under test (SUT). In many MBT methods, these test cases are too abstract to be executed. Therefore, an executable representation of them is necessary to test the SUT. So far, the MBT community has focused on methods that automate the generation of test cases, but less has been done in making them executable. In this paper we propose a language to specify rules that can be automatically applied to produce an executable representation of test cases generated by the Test Template Framework (TTF), a MBT method for the Z notation. 1 The Process of Model-Based Testing Model-based testing (MBT) is a well-known technique aimed at testing software by analysing a formal model or specification of the system under test (SUT) [1, 2]. These techniques have been developed and applied to models written in different formal notations such as Z [3], finite state machines and their extensions [4], B [5], algebraic specifications [6], and so on. The fundamental hypothesis behind MBT is that, as a program is correct if it satisfies its specification, then the specification is an excellent source of test cases. Figure 1 depicts a possible testing process when a MBT method is applied. So far, the MBT community has focused on the “Generation” step in which testers analyse a model of the SUT and generate test cases by applying different techniques. Test cases produced by the “Generation” step are abstract in the sense that they are written in the same language of the model, making them, in most of the MBT methods, not executable. In effect, during the “Refinement” step these abstract test cases are made executable by a process that can be called refinement, concretization or reification. Note that this not necessarily means that the SUT has been refined from the model; it only says that test cases must be refined. In fact, Hierons and others conclude that the relation between refinement and MBT is still a challenge that would have a very tangible benefit if solved [2]. Besides, test case refinement can require an effort equal to the 25% up to 100% of the time spent on modelling [1], so it is worth to automate this step