Regularizing neural networks is an important task to reduce overfitting. Dropout [1] has been a widely-used regularization trick for neural networks. In convolutional neural networks (CNNs), dropout is usually applied to the fully connected layers. Meanwhile, the regularization effect of dropout in the convolutional layers has not been thoroughly analyzed in the literature. In this paper, we an...

We explore a recently proposed Variational Dropout technique that provided an elegant Bayesian interpretation to Gaussian Dropout. We extend Variational Dropout to the case when dropout rates are unbounded, propose a way to reduce the variance of the gradient estimator and report first experimental results with individual dropout rates per weight. Interestingly, it leads to extremely sparse sol...

For modern large-scale Bayesian models, informative priors are difficult if not impossible to elicit. Yet, often, some prior knowledge is known, and this information is incorporated via engineering tricks or methods less principled than a Bayesian prior. However, employing these tricks is difficult to reconcile with principled probabilistic inference. For instance, in the case of data set augme...

Dropout is a method that prevents overfitting when training deep neural networks. It involves sampling different sub-networks by temporarily removing nodes at random. Dropout works well in practice, but its properties have not been fully explored or theoretically justified. Our project explores the properties of dropout by applying methods used in optimization such as simulated annealing and lo...

Dropout is a common problem in the treatment of psychiatric illnesses including bipolar disorders (BD). The aim of the present study is to investigate illness perceptions of dropout patients with BD. A cross sectional study was done on the participants who attended the Mood Disorder Outpatient Clinic at least 3 times from January 2003 through June 2008, and then failed to attend clinic till to ...

Dropout from school hinders development as it makes human potentialities unexplored. This creates all round backwardness. Muslim communities are more backward particularly in Malda district, W.B., where the dropout rate is high. Parental decision to dropout their wards from primary education does not depend upon gender, but depend upon their economic status. Poverty influences the parents to wi...

We show how to adjust for the variance introduced by dropout with corrections to weight initialization and Batch Normalization, yielding higher accuracy. Though dropout can preserve the expected input to a neuron between train and test, the variance of the input differs. We thus propose a new weight initialization by correcting for the influence of dropout rates and an arbitrary nonlinearity’s ...

OBJECTIVE The aim of this study was to use pretreatment and treatment factors to predict dropout from residential substance use disorder program and to examine how the treatment environment modifies the risk for dropout. METHOD This study assessed 3649 male patients at entry to residential substance use disorder treatment and obtained information about their perceptions of the treatment envir...

Regularization for matrix factorization (MF) and approximation problems has been carried out in many different ways. Due to its popularity in deep learning, dropout has been applied also for this class of problems. Despite its solid empirical performance, the theoretical properties of dropout as a regularizer remain quite elusive for this class of problems. In this paper, we present a theoretic...