Unparsing Expressions with Prefix and Postfix Operators

and concrete syntax for infix operators Type rator represents a binary infix operator, which has a text representation, a precedence, and an associativity: hinfixi type precedence = int datatype associativity = LEFT | RIGHT | NONASSOC type rator = string * precedence * associativity This ML code uses simple integers (int) to represent precedence, an enumeration to represent associativity, and a triple to represent an operator. (In the context of an ML type definition, a * does not represent multiplication; it connects elements of a tuple.) The more general unparser, presented below, shows how to use an arbitrary type, not just string, as an operator’s concrete representation. Precedence and associativity determine how infix expressions are parsed into trees, or equivalently, how they are parenthesized. For example, if operator has higher precedence than operator , then x y z= x (y z) and x y z = (x y) z. When two operators have the same precedence, associativity is used to disambiguate. If is left-associative, x y z = (x y) z; if it is right-associative, x y z= x (y z). Some languages have non-associative operators; if is non-associative, then (x y) z 6= x y z 6= x (y z); and x y z may be illegal. The comma that separates parameters in a C function call is an example of a non-associative operator. Many expressions are obtained by applying operators to other expressions, but there must always be indivisible constituents of expressions. We call such constituents atoms. They appear at the leaves of abstract syntax trees and as the non-operator tokens in concrete syntax. In most languages the atoms UNPARSING WITH PREFIX AND POSTFIX OPERATORS 5 include identifiers, integer and real literals, string literals, and so on. In our initial, simple model, an expression is either an atom or an infix operator applied to two expressions: hinfixi+ datatype ast = ATOM of string | APP of ast * rator * ast Type ast represents an expression’s abstract syntax tree. This ML datatype definition is something like a production in a context-free grammar; it gives two alternative ways of constructing an ast. These alternatives are always given names (ATOM and APP), which are called constructors. An ML datatype may contain any number of constructors, and there may be data associated with each one. Here, for an atom, we care only about the atom’s string representation. For an application, we want the operator and the asts representing the two operands. Because ast is used in its own definition, it is a recursive type, and values of type ast are trees. Type type ast represents the input to the unparser; we also need a type to represent the unparser’s output. For simplicity, we treat this output as a sequence of lexemes, where a lexeme represents an atom, an operator, or a parenthesis. Moreover, we undertake to emit only sequences in which atoms and operators alternate, or in which parenthesized sequences take the place of atoms. Let us call such a sequence an image and use a representation that forces it to satisfy the following grammar: image ) lexical-atom operator lexical-atom lexical-atom ) atom ( image ) In the corresponding ML, the sequence in braces becomes the type image’: hinfixi+ datatype image = IMAGE of lexical_atom * image’ and image’ = EOI (* end of image *) | INFIX of rator * lexical_atom * image’ and lexical_atom = LEX_ATOM of string | PARENS of image EOI represents the end of the image, and PARENS represents an image in parentheses. This representation enforces the invariants that expressions and operators alternate, and that the first and last elements of an image are expressions. Parsing infix expressions To be correct, an unparser must produce a sequence that parses back into the original abstract syntax tree. We develop an unparsing algorithm by thinking about parsing. To understand how to minimize parentheses, we need to consider where parentheses are needed to get the correct parse. Suppose we have an abstract syntax tree that has been obtained by parsing an image without any parentheses. Then wherever the syntax tree has an APP node whose parent is also an APP node, there are two cases: the child may be the left child or the right child of its parent: