Bayesian Backpropagation Over I-O Functions Rather Than Weights
نویسنده
چکیده
FIGURES Figures 1 through 3: Train using unmodified BP on training set t, and feed input x into the resultant net. The horizontal axis gives the output you get. If t and x were still used but training had been with modified BP, the output would have been the value on the vertical axis. In succession, the three figures have α = .6, .4, .4, and m = 1, 4, 1. Figure 3. Figure 4: The horizontal axis is |w i |. The top curve depicts the weight decay regularizer, αw i 2 , and the bottom curve shows that regu-larizer modified by the correction term. α = .2.
منابع مشابه
Bayesian Backpropagation Over 1-0 Functions Rather Than Weights
The conventional Bayesian justification of backprop is that it finds the MAP weight vector. As this paper shows, to find the MAP i-o function instead one must add a correction tenn to backprop. That tenn biases one towards i-o functions with small description lengths, and in particular favors (some kinds of) feature-selection, pruning, and weight-sharing.
متن کاملA Bayesian approach for initialization of weights in backpropagation neural net with application to character recognition
Convergence rate of training algorithms for neural networks is heavily affected by initialization of weights. In this paper, an original algorithm for initialization of weights in backpropagation neural net is presented with application to character recognition. The initialization method is mainly based on a customization of the Kalman filter, translating it into Bayesian statistics terms. A me...
متن کاملA Practical Bayesian Framework for Backprop Networks
A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible: (1) objective comparisons between solutions using alternative network architectures; (2) objective stopping rules for deletion of weights; (3) objective choice of magnitude and type of weight decay terms or additive regularisers (for penalising large weigh...
متن کاملAssumed Density Filtering Methods for Learning Bayesian Neural Networks
Buoyed by the success of deep multilayer neural networks, there is renewed interest in scalable learning of Bayesian neural networks. Here, we study algorithms that utilize recent advances in Bayesian inference to efficiently learn distributions over network weights. In particular, we focus on recently proposed assumed density filtering based methods for learning Bayesian neural networks – Expe...
متن کاملNatural-Parameter Networks: A Class of Probabilistic Neural Networks
Neural networks (NN) have achieved state-of-the-art performance in various applications. Unfortunately in applications where training data is insufficient, they are often prone to overfitting. One effective way to alleviate this problem is to exploit the Bayesian approach by using Bayesian neural networks (BNN). Another shortcoming of NN is the lack of flexibility to customize different distrib...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 1993