Adaboost is a machine learning algorithm that builds a series of small decision trees, adapting each tree to predict difficult cases missed by the previous trees and combining all trees into a single model. We will discuss the AdaBoost methodology and introduce the extension called Real AdaBoost. Real AdaBoost comes from a strong academic pedigree: its authors are pioneers of machine learning a...

Recently ensemble methods like AdaBoost were successfully applied to character recognition tasks, seemingly defying the problems of overrtting. This paper shows that although AdaBoost rarely overrts in the low noise regime it clearly does so for higher noise levels. Central for understanding this fact is the margin distribution and we nd that AdaBoost achieves { doing gradient descent in an err...

Semi-supervised Learning is a machine learning approach that, by making use of both labeled and unlabeled data for training, can significantly improve learning accuracy. Boosting is a machine learning technique that combines several weak classifiers to improve the overall accuracy. At each iteration, the algorithm changes the weights of the examples and builds an additional classifier. A well k...

In many problems of machine learning and computer vision, there exists side information, i.e., information contained in the training data and not available in the testing phase. This motivates the recent development of a new learning approach known as learning with side information that aims to incorporate side information for improved learning algorithms. In this work, we describe a new traini...

In an earlier paper, we introduced a new “boosting” algorithm called AdaBoost which, theoretically, can be used to significantly reduce the error of any learning algorithm that consistently generates classifiers whose performance is a little better than random guessing. We also introduced the related notion of a “pseudo-loss” which is a method for forcing a learning algorithm of multi-label con...

We construct a framework which allows an algorithm to turn the distributions produced by some boosting algorithms into polynomially smooth distributions (w.r.t. the PAC oracle’s distribution), with minimal performance loss. Further, we explore the case of Freund and Schapire’s AdaBoost algorithm, bounding its distributions to polynomially smooth. The main advantage of AdaBoost over other boosti...

”Boosting” is a general method for improving the performance of any weak learning algorithm that consistently generates classifiers which need to perform only slightly better than random guessing. A recently proposed and very promising boosting algorithm is AdaBoost [4]. It has been applied with great success to several benchmark machine learning problems using rather simple learning algorithms...

Real Adaboost is a well-known and good performance boosting method used to build machine ensembles for classification. Considering that its emphasis function can be decomposed in two factors that pay separated attention to sample errors and to their proximity to the classification border, a generalized emphasis function that combines both components by means of a selectable parameter, l, is pre...

Boosting is known to be sensitive to label noise. We studied two approaches to improve AdaBoost’s robustness against labelling errors. One is to employ a label-noise robust classifier as a base learner, while the other is to modify the AdaBoost algorithm to be more robust. Empirical evaluation shows that a committee of robust classifiers, although converges faster than non label-noise aware Ada...

Methods that create several classifiers out of one base classifier, so-called ensemble creation methods, have been proposed and successfully applied to many classification problems recently. One category of such methods is Boosting with AdaBoost being the best known procedure belonging to this category. Boosting algorithms were first developed for two-class problems, but then extended to deal w...