Modular Reasoning in Object-Oriented Programming
چکیده
Difficulties in reasoning about functional correctness and relational properties of object-oriented programs are reviewed. An approach using auxiliary state is briefly described, with emphasis on the author’s work. Some near term challenges are sketched. Formal verification depends on scientific theories of programming, which answer questions such as these: What are good models of computational behavior? What behavioral properties of components are needed for modular reasoning about a composed system? How can such properties be specified and a component be verified, or even derived from its specification? How can a program and justification of its correctness be revised in accord with small revision of its specification? Such questions have well developed answers that are adequate for small programs under strong simplifying assumptions. But many useful programs are quite large and built from complicated components that violate simplifying assumptions. The longstanding challenge of compositional reasoning remains substantially unsolved. Object-oriented programs pose several challenges that are the focus of my recent research, in which auxiliary state is being used to specify encapsulation boundaries and disciplined interdependence. Section 2, explains the approach, accomplishments, and challenges in terms of invariants for shared mutable objects. Section 3 addresses relational properties including data refinement and secure information flow. This line of research has been carried out for Java-like programming languages; I argue in Section 1 for the importance of such languages. Some additional challenges pertinent to objectoriented programming, but not tied to the main theme, are discussed in Section 4. A detailed tutorial on the state-based approach to encapsulation advocated here appears elsewhere [33]. Several near-term challenges (1–5 years) are presented here in the setting of sequential object-oriented programs. Because the approach taken here is based on the use of assertions, it is also quite relevant to verification of concurrent object-oriented programs and low level imperative code. 1 Why Java-like language? In order to develop theory for modular reasoning about large programs, we need a corpus of large programs and automated support for experiments. Since I would like to do ? Partially supported by the National Science Foundation under grants CCR-0208984 and CCF0429894 and by Microsoft Research. science that contributes to human good through improved engineering, the primary objects of study should be representative examples of large programs that are significantly deployed and used. This means confronting programs written in notations like C, Java, and C#—though not necessarily handling all of their features without restriction. Aside from obvious pragmatic reasons for interest in Java-like languages, there are technical reasons why such a language is a good point in the language design space. – The language is sufficiently rich to express higher order design patterns which are needed for well structured programs and used in common practice. – Despite the preceding item, the language is essentially “defunctionalized” [42, 4] owing to the binding of methods to classes rather than to instances. Thus relatively simple semantic models are adequate, at least for large fragments of the language. For example, my work discussed in Sections 2 and 3 has been done using a straightforward Scott-Strachey denotational semantics, for a fragment of Java including recursive types, inheritance, mutable objects, and other features without restriction; this model has been encoded in PVS [34]. Nipkow’s group and others have obtained strong results using straightforward operational models [26]. – The module system (packages, generic classes, public/private/protected visibility) embodies most of what current theory offers for scope-based encapsulation. – The Java type system is name-based; named types provide a convenient hook on which to hang specifications and encapsulation boundaries. In particular, it helps deal with inheritance, which is widely used if problemmatic. – Pointer arithmetic is absent. Parameter passing is by value and identifiers cannot alias. Method declarations are not nested, avoiding the semantic complexity of reference to variables in enclosing scopes other than global scope.1 These features are not without cost. Java programs make much use of global variables (“statics”)—global in that they are in outermost scopes; this is mitigated in that the scope of visibility may be a single class or package. Reflection, at least in full generality, is a feature I see as a very difficult and long-term challenge for verification. This is exacerbated in that reflection, like threads and permission-based access control, appears in the form of special libraries rather than being distinguished with separate syntax. Perhaps the highest cost is the ubiquity of aliasing in the sense of shared references to mutable objects in the heap. 2 Heap encapsulation using auxiliary state For modular reasoning in object-oriented programming there are several challenges. 1. Non-hierarchical control flow due to callbacks leads, even in sequential programs, to interference like that in concurrent programs. 2. The conventional notion of layered abstraction is also subverted by non-hierarchical control flow due to inheritance and method overriding. 1 Compare the complexity of Idealized Algol models [44] with Modula-3 and Oberon, where non-local references are restricted for those procedures that are passed as arguments or stored in variables [32]. 3. Design patterns that are essentially higher order are often used, but unlike in functional programming the encapsulation aspects are not explicit in the program text, owing to data representation based on shared heap objects. 4. Functional aspects of such patterns are also not specified formally, for lack of good models (compare “map” in functional programming with the “Visitor” pattern). The second challenge is addressed by the notion of behavioral subtyping which is well understood [29, 20] except that the extant theories do not fully deal with the first and third challenges. For the fourth challenge, which we discuss in Section 4, one might argue that at best we should aim for verifying simple safety properties. Indeed, in his VSTTE talk Bart Jacobs said that full functional verification of nontrivial Java programs is impractical. But for realistically complex systems, attempts to verify simple safety properties lead to the need for more general properties, especially object invariants. For the first and third challenges, progress is being made using auxiliary state to express encapsulation using assertions. That is the topic of this section, which focuses on object invariants. More extensive discussions and citations on these topics can be found in Müller’s VSTTE paper [31] and my survey paper [33]. Non-hierarchical control flow. As an example of the first challenge, consider a sensor playing the role of Subject in the Subject/Observer pattern [22]. The sensor maintains a set of registered Views: when the sensor value reaches the threshhold v.thresh of a given view v, the sensor invokes method v.notify and removes v from the set. This description is in terms of a set, part of the abstraction offered by the Subject; the implementation might store views in an array ordered by thresh values. The pattern cannot be seen simply as a client using an abstraction, because notify is what is known as an upcall to the client. The difficulty is that v.notify may make a reentrant callback to the sensor. Some callbacks are quite sensible, e.g., the view could query the sensor value. But trouble is likely if v.notify invokes a method to enumerate the current set of views. While notifications are under way, the array may be in an inconsistent state—is v in the set? in the array?—yet the enumeration method may assume as precondition the sensor’s invariant. Non-hierarchical control flow renders naive reasoning about object invariants unsound. The problem is similar to interference in shared-variable concurrency, for which there are several established and well understood solutions. For the reentrant callback problem, which already occurs in sequential code, the situation is less settled, although the probem is a frequent cause of insidious bugs. Various solutions have been proposed: – Establish caller’s invariant before everymethod call. But this is impractical in many cases: most calls do not result in reentrant callbacks and good use of abstraction in design leads to many calls to substructures while a super-structure’s invariant is temporarily violated. – Use concurrency locks. But this leads to deadlocks in the sequential case. – Use temporal specification of allowed calling sequences. This can be heavy handed and violates abstraction by making method calls visible. Moreover, verification of such properties requires the whole program in general. A more promising approach begins by making the invariant an explicit precondition on those methods that assume it, like the enumerator in the example. This precondition cannot be established by client v attempting a reentrant callback, unless in fact the sensor restores its invariant before invoking v.notify. An object invariant I ought not appear in the precondition of a public method, as that could expose the internal representation. Various techniques have been proposed to hide information, e.g., treating I in a precondition as an opaque predicate [14, 15], a typestate [19], a call to a pure method, or a model field [30, 25]. We advocate the approach of Leino et al [8], known as the Boogie methodology or the inv/own discipline. We give a simplified account sufficient for discussion. The discipline uses a ghost (auxiliary) field2 inv of type boolean which represents whether the invariant of o is in force, just as a programmer might do using an ordinary field. There are several associated proof obligations; together they embody a discipline that ensures the following is a program invariant, i.e., it holds in all reachable states: (∀ o | o.inv ⇒ I(o) ) (1) Informally: for each allocated object o, the object’s invariant holds if o.inv = true. Thus within the body of a method with precondition inv, one can exploit the invariant I while exposing to clients not the predicate I but only the boolean field inv. Heap encapsulation. Besides its own fields, an object may depend on some objects that serve as its internal representation. This can be represented using another auxiliary field by which an object points to its direct owner, if any. An object’s invariant is allowed to depend only on objects it transitively owns. An associated program invariant is that o.inv implies p.inv for every object p owned by o. If an object is in a consistent state then so are its representation objects. This invariant is maintained owing to a proof obligation: update of a field of an object p has as precondition that p.inv = false. So, if an object p is susceptible to update then not only may I(p) be temporarily violated but also if p is part of the representation of some object o then also o.inv = false and I(o) may be temporarily violated. Ownership imposes a forest structure on the heap, separating encapsulated data from clients. Ownership types [18, 2] embody this idea and an account of the resulting encapsulation has been given in terms of the theory of representation independence [5]. But it has proved difficult to find an ownership type system that admits common design patterns and also enforces encapsulation sufficiently strong for modular reasoning about object invariants. In particular, many examples call for the transfer of ownership (e.g., in resource management) and this does not sit well with types. An alternative to types is separation logic [45, 39]. In separation logic, owning an object p has been equated with having a precondition dependent on p. A modest challenge is how to scale the logic up to classes (instantiable abstractions) instead of singleinstance modules. A bigger challenge is how to cope with the fact that in object-oriented 2 For our purposes, a model field is an auxiliary field, the value of which is defined as a function of other state, whereas the value of a ghost field must be updated by explicit auxiliary assignments. languages, the object is the unit of addressability but some fields are inherited and others (to be added in subclasses) are not known to the modular reasoner. Parkinson and Bierman [41, 14] have taken initial steps and their treatment of encapsulation has been given an acount in terms of higher order separation logic [13, 15]. By contrast with separation logic, the approach described here is compatibility with standard logics and specification notions, which can leverage existing tools and programmer expertise. One advantage of encoding ownership with a ghost field is that transfer is straightforward; the field is mutable. In combination with the invariant-tracking field inv, the discipline [8, 28] expresses very directly the flow of control in and out of hierarchical encapsulation boundaries even as those boundaries are mutated. The most exciting advantage of the approch is that it generalizes to more elaborate patterns. Ownership is concerned with a single object and its representation. Already the pattern of iterators is problemmatic, in that an iterator needs access to the representation objects of its associated collection but a collection is not owned by its iterators. There are many situations where several publically-accessible objects cooperate to provide an abstraction, so their individual invariants need to depend on non-owned objects. Just as the owner field records a dependence that can be taken into account in reasoning, one can use a ghost field to record the dependence between peer objects. This idea has been developed in the simple case of one object’s invariant depending on another: the “friendship” discipline [37, 10] imposes modular obligations on both dependee and dependant, so (1) is maintained even when an invariant I depends on non-owned objects. A field deps is used so that p.deps is a set of object references that includes all o that could have an invariant currently dependent on p that is not licensed by ownership. The friendship discipline has been successfully applied to several design patterns including iterators [38] and Subject/View [10], but it does not seem likely that there is a single such discipline sufficiently general to handle every situation. I believe that by using auxiliary state to record encapsulation boundaries for heap structure, we can formalize a number of generally applicable specification patterns. Interactive theorem proving or just pencil and paper can be used to show that the associated global invariant is a consequent of the pattern’s stipulated annotation discipline. Automated first-order provers may then be used to discharge the assertions in particular instances of the pattern, treating program invariants like (1) as axiom schemes. For patterns that can be specified using just ownership, the Spec# system implements the Boogie methodology using a first-order prover as discussed in the VSTTE paper of Barnett et al [9]. Ownership can also be encoded in the JML specification language which is being used in a number of verification systems, as discussed in the VSTTE paper of Leavens and Clifton [27]. There is impressive agreement about syntax but the semantics is neither formalized nor entirely consistent between projects. Within a 5-year time frame it should be possible to provide a foundational logic for JML, encompassing encapsulation (via scope and via auxiliary state), reentrancy, and behavioral subtyping. This would serve to integrate and assess, in particular helping to ensure that the axiomatic semantics embodied in some tools is sound with respect to an (idealized) operational semantics. Concurrency specification is less developed but a sound treatment using strong atomicity assumptions should be within reach [46]. 3 Relational properties By relational property I mean notions like simulation, where a pair of programs preserve some relation. The most important relational property is preservation of a simulation between implementations of an abstract data type —yielding modular proof of program equivalence or data refinement. Another is noninterference in the sense of secure information flow and dependency analyis [1, 47], which in turn can be used to justify use of impure method calls in specifications [36]. Sampaio et al developed a refinement calculus for a subset of Java [16] and implemented a tool that applies general refactoring transformations that are validated on the basis of a theory of data refinement [17]. The latter theory is only sound in the absence of heap sharing. Anindya Banerjee and I have adapted the inv/own discipline to support representation independence [7], i.e., soundness of simulations for proof of program equivalence with heap sharing. In five years it should be possible to integrate these theories to encompass refinement and shared heap objects, allowing as units of encapsulations multiple classes and more importantly small configurations of cooperating objects (e.g., a set and its enumerators). An associated milestone would be a refactoring tool, say for Java’s Eclipse developments environment, that applies semantically validated transformations. Benton [12] and Yang [50] propose relational Hoare logics in which the basic correctness condition takes the form