First-Class Functions for First-Order Database Engines
چکیده
We describe query defunctionalization which enables off-the-shelf first-order database engines to process queries over first-class functions. Support for first-class functions is characterized by the ability to treat functions like regular data items that can be constructed at query runtime, passed to or returned from other (higher-order) functions, assigned to variables, and stored in persistent data structures. Query defunctionalization is a non-invasive approach that transforms such function-centric queries into the data-centric operations implemented by common query processors. Experiments with XQuery and PL/SQL database systems demonstrate that first-order database engines can faithfully and efficiently support the expressive “functions as data” paradigm. 1. Functions Should be First-Class Since the early working drafts of 2001, XQuery’s syntax and semantics have followed a functional style: functions are applied to form complex expressions in a compositional fashion. The resulting XQuery script’s top-level expression is evaluated to return a sequence of items, i.e., atomic values or XML nodes [8]. Ten years later, with the upcoming World Wide Web Consortium (W3C) XQuery 3.0 Recommendation [28], functions themselves now turn into first-class items. Functions, built-in or user-defined, may be assigned to variables, wrapped in sequences, or supplied as arguments to and returned from higher-order functions. In effect, XQuery finally becomes a full-fledged functional language. Many useful idioms are concisely expressed in this “functions as data” paradigm. We provide examples below and argue that support for first-class functions benefits other database languages, PL/SQL in particular, as well. This marks a significant change for query language implementations, specifically for those built on top of (or right into) database kernels. While atomic values, sequences, or XML nodes are readily represented in terms of the widespread first-order database data models [9], this is less obvious for function items. Database kernels typically lack a runtime representation of functional values at all. We address this challenge in the present work. In query languages, the “functions as data” principle can surface in various forms. Functions as Values. XQuery 3.0 introduces name#n as notation to refer to the n-ary function named name: math:pow#2 refers to exponentiation while fn:concat#2 denotes string concatenation, for example. The values of these expressions are functions—their types are of the form function(t1) as t2 or, more succinctly, t1 → t2—which may be bound to variables and applied to arguments. The evaluation of the expression let $exp := math:pow#2 return $exp(2,3) yields 8, for example. 1“[. . . ] XQuery is a functional language in which a query is represented as an expression.” [11, §2] 1 declare function fold-right( 2 $f as function(item(), item()*) as item()*, 3 $z as item()*, $seq as item()*) as item()* 4 { 5 if (empty($seq)) then $z else 6 $f(fn:head($seq), fold-right($f, $z, fn:tail($seq))) 7 }; Figure 1: Higher-order function fold-right (XQuery 3.0). Higher-Order Functions. In their role of regular values, functions may be supplied as parameters to and returned from other functions. The latter, higher-order functions can capture recurring patterns of computation and thus make for ideal building blocks in query library designs. Higher-order function fold-right is a prime example here—entire query language designs have been based on its versatility [13, 18]. The XQuery 3.0 variant fold-right($f, $z, $seq) is defined in Figure 1: it reduces a given input sequence $seq = (e1,e2,. . . ,en) to the value $f(e1,$f(e2,$f(. . . ,$f(en,$z)· · · ))). Different choices for the functional parameter $f and $z configure fold-right to perform a variety of computations: fold-right(math:pow#2, 1, (e1,e2,. . . ,en)) (with numeric ei) computes the exponentiation tower e en . . . e2 1 , while the expression fold-right(fn:concat#2, "", (e1,e2,. . . ,en)) will return the concatenation of the n strings ei. Function Literals. Queries may use function(x) { e } to denote a literal function (also: inline function or λ-expression λx.e). Much like the literals of regular first-order types (numbers, strings, . . . ), function literals are pervasive if we adopt a functional mindset: A map, or associative array, is a function from keys to values. Figure 2 takes this definition literally and implements maps in terms of functions. Empty maps (created by map:empty) are functions that, for any key $x, will return the empty result (). A map with entry ($k,$v) is a function that yields $v if a key $x = $k is looked up (and otherwise will continue to look for $x in the residual map $map). Finally, map:new($es) builds a complex map from a sequence of entries $es—an entry is added through application to the residual map built so far. As a consequence of this implementation in terms of functions, lookups are idiomatically performed by applying a map to a key, i.e., we may write let $m := map:new((map:entry(1,"one"), map:entry(2,"two"))) return $m(2) (: "two" :) 2Our design follows Michael Kay’s proposal for maps in XSLT 3.0. Of two entries under the same key, we return the entry inserted first (this is implementation-dependent: http://www.w3.org/TR/xslt-30/#map). ar X iv :1 30 8. 01 58 v1 [ cs .D B ] 1 A ug 2 01 3 1 declare function map:empty() { 2 function($x) { () } 3 }; 4 5 declare function map:entry($k,$v) { 6 function($map) { 7 function($x) { if ($x = $k) then $v else $map($x) } 8 } 9 }; 10 11 declare function map:new($es) { 12 fold-right(function($f,$x) { $f($x) }, map:empty(), $es) 13 }; 14 15 declare function map:remove($map,$k) { 16 function($x) { if ($x = $k) then () else $map($x) } 17 }; Figure 2: Direct implementation of maps as functions from keys to values (XQuery 3.0). 1 (: wrap(), unwrap(): see Section 4.1 :)