This lecture is the first of three lectures investigating the complexity of neural networks. We’ll cover 3 bounds, 2 of which use covering numbers, and at the end point show how to relate them to Rademacher complexity. The definition of cover we’ll use is as follows. Definition. Say G is a (‖ · ‖p; , S)-cover of F if: • For every f ∈ F , there exists g ∈ G so that ‖g(S)− f(S)‖p ≤ , where g(S) :...

Let G(n ;m) denote a graph of n vertices and m edges. Vertices of G will be denoted by x 1 ,. .. ,y 1. .. ; edges will be denoted by (x, y) and triangles by (x, y, z). (G-x 1-x 2 x k) will denote the graph G from which the vertices x 1 ,. .. , xk and all edges incident to them have been omitted. has been omitted. A special case of a well known theorem of Turin states 2 that every G(n ; 4 + 1) c...

The sample complexity of learning a Boolean-valued function class is precisely characterized by its Rademacher complexity. This has little bearing, however, on the sample complexity of efficient agnostic learning. We introduce refutation complexity, a natural computational analog of Rademacher complexity of a Boolean concept class and show that it exactly characterizes the sample complexity of ...

Let E be a separable (or the dual of a separable) symmetric function space, let M be a semifinite von Neumann algebra and let E(M) be the associated noncommutative function space. Let (εk)k≥1 be a Rademacher sequence, on some probability space Ω. For finite sequences (xk)k≥1 of E(M), we consider the Rademacher averages ∑ k εk ⊗ xk as elements of the noncommutative function space E(L(Ω)⊗M) and s...

The sample complexity of learning a Boolean-valued function class is precisely characterized by its Rademacher complexity. This has little bearing, however, on the sample complexity of efficient agnostic learning. We introduce refutation complexity, a natural computational analog of Rademacher complexity of a Boolean concept class and show that it exactly characterizes the sample complexity of ...

We show that a Rademacher expansion can be used to establish an exact bound for the entropy of black holes within a conformal field theory framework. This convergent expansion includes all subleading corrections to the Bekenstein-Hawking term.

We consider regression with square loss and general classes of functions without the boundedness assumption. We introduce a notion of offset Rademacher complexity that provides a transparent way to study localization both in expectation and in high probability. For any (possibly non-convex) class, the excess loss of a two-step estimator is shown to be upper bounded by this offset complexity thr...

A generic way to extend generalization bounds for binary large-margin classifiers to large-margin multi-category classifiers is presented. The simple proceeding leads to surprisingly tight bounds showing the same Õ(d) scaling in the number d of classes as state-of-the-art results. The approach is exemplified by extending a textbook bound based on Rademacher complexity, which leads to a multi-cl...

This paper presents several novel generalization bounds for the problem of learning kernels based on the analysis of the Rademacher complexity of the corresponding hypothesis sets. Our bound for learning kernels with a convex combination of p base kernels has only a log p dependency on the number of kernels, p, which is considerably more favorable than the previous best bound given for the same...