The Vapnik-Chervonenkis dimension is a combinatorial parameter that reflects the ”complexity” of a set of sets (a.k.a. concept classes). It has been introduced by Vapnik and Chervonenkis in their seminal paper [1] and has since found many applications, most notably in machine learning theory and in computational geometry. Arguably the most influential consequence of the VC analysis is the funda...

Learning Theory studies learning objects from random samples. The main question is: How many samples do we need to ensure an error bound with certain con ̄dence? To answer this question, the covering numbers or entropy numbers play an essential role, as shown by Vapnik, Poggio, CuckerSmale, and many others. For kernel machine learning such as the Support Vector Machine, a Reproducing Kernel Hilb...

The field of optimization and machine learning are increasingly interplayed and optimization in different problems leads to the use of machine learning approaches‎. ‎Machine learning algorithms work in reasonable computational time for specific classes of problems and have important role in extracting knowledge from large amount of data‎. ‎In this paper‎, ‎a methodology has been employed to opt...

We report a novel possibility for extracting a small subset of a data base which contains all the information necessary to solve a given classification task: using the Support Vector Algo rithm to train three different types of handwritten digit classifiers, we observed that these types of classifiers construct their decision surface from strongly overlapping small (k: 4%) subsets of the data b...

We present an overview of learning theory including its statistical and computational aspects. We start by giving a probabilistic formulation of learning problems including classification and regression where the learner knows nothing about the probability distribution generating the data. We then consider the principle of empirical risk minimization (ERM) that chooses a function from a given c...