Hands-On Learning Theory Fall 2016, Lecture 4
نویسنده
چکیده
Recall that in Theorem 2.1, we analyzed empirical risk minimization with a finite hypothesis class F , i.e., |F| < +∞. Here, we will prove results for possibly infinite hypothesis classes. Although the PAC-Bayes framework is far more general, we will concentrate of the prediction problem as before, i.e., (∀f ∈ F) f : X → Y. Also, note that Theorem 2.1 could have been stated in a more general fashion and not only for the 0/1 risk 1[f(x) 6= y]. Here, we will use a more general distortion function d : Y × Y → [0, 1]. Under this setting, the 0/1 risk is given by d(y, y′) = 1[y 6= y′]. Compared to Theorem 2.1, in the PAC-Bayes setting, instead of choosing a single predictor f , the learner picks a distribution Q. (It should be clear that the latter generalizes the previous setting.) Fix a prior distribution P of support F . After observing a training set of n samples, the task of the learner is to choose a posterior distribution Q of support F . PAC-Bayes guarantees are given with respect to a prior distribution P and simultaneously for all posterior distributions Q. By looking at the following theorem statement, the reader would also notice the relationship between PAC-Bayes and KL-regularization.
منابع مشابه
Hands-On Learning Theory Fall 2016, Lecture 3
A basic property of the entropy of a discrete random variable x is that: 0 ≤ H(x) ≤ log |X | In fact, the entropy is maximal for the discrete uniform distribution. That is, (∀x ∈ X ) p(x) = 1/|X |, in which case H(x) = log |X |. Definition 3.2 (Conditional entropy). The conditional entropy of y given x is defined as: H(y|x) = ∑ v∈X px(v)H(y|x = v) = − ∑ v∈X px(v) ∑ y∈Y py|x(y|v) log py|x(y|v) =...
متن کاملImproving Students’ Learning Outcomes in Safety Education through Interdepartmental Collaboration
It is critical to learn the theory to work safely. However, it is also important to learn how to apply such theories into the workplace. In addition, adults learn best when they are active partners in the learning process. The authors emphasize not to lecture adult learners but rather, get them involved in discussions, problem solving and hands-on activities. To assist this effort, Purdue Unive...
متن کاملProbability Theory 1 Lecture Notes
These lecture notes were written for MATH 6710 at Cornell University in the Fall semester of 2013. They were revised in the Fall of 2015 and the schedule on the following page re ects that semester. These notes are for personal educational use only and are not to be published or redistributed. Almost all of the material, much of the structure, and some of the language comes directly from the co...
متن کاملHow We Teach Student responses to a hands-on kinesthetic lecture activity for learning about the oxygen carrying capacity of blood
Breckler J, Yu JR. Student responses to a hands-on kinesthetic lecture activity for learning about the oxygen carrying capacity of blood. Adv Physiol Educ 35: 39–47, 2011; doi:10.1152/advan.00090.2010.—This article describes a new hands-on, or “kinesthetic,” activity for use in a physiology lecture hall to help students comprehend an important concept in cardiopulmonary physiology known as oxyg...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2017