The 2005 AAAI Classic Paper Awards

approximately correct learning). At the same time, his paper and his subsequent research applied that theoretical framework to analyze the properties of specific machine learning algorithms. At the time of Haussler’s paper, one informal notion popular in machine learning was the “inductive bias” of a learner; that is, the set of assumptions that, together with the training data, logically entail the hypothesis finally output by the learner. For example, a key part of the inductive bias of most learners is the representation they employ for hypotheses, which defines implicitly the space of hypotheses they can ever consider. The larger the space of hypotheses considered, the more training data needed to reliably learn the target concept—the more constraining the inductive bias, the less training data needed. During the mid-1980s the new theory of PAC learning was being developed, which allowed deriving quantitative bounds on the probability of successful learning as a function of the number of training examples and the complexity of the learner’s hypothesis space (as measured by its VapnikChervonenkis dimension). What Haussler’s paper did was help introduce this PAC learning theory to the AI community and show that its results made direct contact with the inductive bias of the learner—that in fact the Vapnik-Chervonenkis dimen■ Mitchell and Levesque provide commentary on the two AAAI Classic Paper awards, given at the AAAI-05 conference in Pittsburgh, Pennsylvania. The two winning papers were “Quantifying the Inductive Bias in Concept Learning,” by David Haussler, and “Default Reasoning, Nonmonotonic Logics, and the Frame Problem,” by Steve Hanks and Drew McDermott.