Application of Optimal Search Theory to ILP

For most ILP systems that attempt to find a hypothesis by searching through a space of possible alternatives, the principal problem is that the size of the search space makes it impossible to perform even a complete (but non-exhaustive) search. Hence, greedy approaches such as FOIL [1] and restricted variants of branch and bound techniques (Aleph [2]), etc. are used to restrict the search space. In this paper, we will concern ourselves principally with systems like Progol [3], in which search proceeds by examining subspaces constrained by a set of most-specific clause ⊥1,⊥2, . . . ,⊥n (where n here will be the number of positive examples) using a greedy covering procedure. In each iteration of the greedy procedure a Progol-like system selects a positive example ei, construct a bottom clause ⊥i using the example and the background knowledge B and then searches for the best clause hi in a lattice lat(⊥i) subsuming ⊥i. Here “best” is in terms of some objective function such as pos(hi)− neg(hi)− size(hi), where pos and neg are functions that compute the positive and negative examples covered by a clause hi and size is a function that computes some measure of clause size. Once this best clause hi in the lattice lat(⊥i) is found, all positive examples made redundant by B ∧ hi are removed from further consideration; and the search continues. A number of questions still remain, for example: How is the ei to be selected? How much effort should we expend on any one search? Current ILP systems answer these in ad hoc ways. It is our aim to show how a theory of optimal search described in [4] can be applied to answer such questions.