Classical and recent results in statistical pattern recognition and learning theory are reviewed in a two-class pattern classification setting. This basic model best illustrates intuition and analysis techniques while still containing the essential features and serving as a prototype for many applications. Topics discussed include nearest neighbor, kernel, and histogram methods, Vapnik–Chervone...

Vapnik-Chervonenkis (VC) bounds play an important role in statistical learning theory as they are the fundamental result which explains the generalization ability of learning machines. There have been consequent mathematical works on the improvement of VC rates of convergence of empirical means to their expectations over the years. The result obtained by Talagrand in 1994 seems to provide more ...

N. Bredeche, S. Gelly, M. Schoenauer, O. Teytaud. A Statistical Learning Approach to bloat and universal consistency in genetic programming. Poster of Gecco 2005. S. Gelly, O. Teytaud, N. Bredeche, M. Schoenauer. Apprentissage statistique et programmation genetique : la croissance du code est-elle inevitable ? pp163-178. Proceedings of CAP’2005. Universal Consistency, the convergence to the min...

Piecewise Linear-Quadratic (PLQ) penalties are widely used to develop models in statistical inference, signal processing, and machine learning. Common examples of PLQ include least squares, Huber, Vapnik, 1-norm, their asymmetric generalizations. Properties these estimators depend on the choice penalty its shape parameters, such as degree asymmetry for quantile loss, transition point between li...

We study annealed theories of learning boolean functions using a concept class of nite cardinality The naive annealed theory can be used to derive a univer sal learning curve bound for zero temperature learning similar to the inverse square root bound from the Vapnik Chervonenkis theory Tighter nonuniversal learning curve bounds are also derived A more re ned annealed theory leads to still tigh...