Rademacher and Gaussian complexities are successfully used in learning theory for measuring the capacity of the class of functions to be learned. One of the most important properties for these complexities is their Lipschitz property: a composition of a class of functions with a fixed Lipschitz function may increase its complexity by at most twice the Lipschitz constant. The proof of this prope...

We suggest a penalty function to be used in various problems of structural risk minimization. This penalty is data dependent and is based on the sup-norm of the so called Rademacher process indexed by the underlying class of functions (sets). The standard complexity penalties, used in learning problems and based on the VCdimensions of the classes, are conservative upper bounds (in a probabilist...

In 1939 Rademacher demonstrated how to express Klein’s modular invariant as a sum over elements of the modular group. In this article we generalize Rademacher’s approach so as to construct bases for the spaces of automorphic integrals of arbitrary even integer weight, for an arbitrary group commensurable with the modular group. Our methods provide explicit expressions for the Fourier expansions...

For a finite function class we describe the large sample limit of the sequential Rademacher complexity in terms of the viscosity solution of a G-heat equation. In the language of Peng’s sublinear expectation theory, the same quantity equals to the expected value of the largest order statistics of a multidimensional G-normal random variable. We illustrate this result by deriving upper and lower ...

In this paper, we analyze the properties of cross-validation from the perspective of the stability, that is, the difference between the training error and the error of the selected model applied to any other finite sample. In both the i.i.d. and non-i.i.d. cases, we derive the upper bounds of the one-round and average test error, referred to as the one-round/convoluted Rademacher-bounds, to qua...

In this lecture, we discuss Rademacher complexity, which is a different (and often better) way to obtain generalization bounds for learning hypothesis classes.