The restricted Boltzmann machine (RBM) has received an increasing amount of interest in recent years. It determines good mapping weights that capture useful latent features in an unsupervised manner. The RBM and its generalizations have been successfully applied to a variety of image classification and speech recognition tasks. However, most of the existing RBM-based models disregard the preser...

We present a mathematical construction for the restricted Boltzmann machine (RBM) that does not require specifying the number of hidden units. In fact, the hidden layer size is adaptive and can grow during training. This is obtained by first extending the RBM to be sensitive to the ordering of its hidden units. Then, with a carefully chosen definition of the energy function, we show that the li...

We explore the training and usage of the Restricted Boltzmann Machine for unsupervised feature extraction. We investigate the many different aspects involved in their training, and by applying the concept of iterate averaging we show that it is possible to greatly improve on state of the art algorithms. We also derive estimators based on the principles of pseudo-likelihood, ratio matching, and ...

Recent developments have demonstrated deep models to be very powerful generative models which are able to extract features automatically and obtain high predictive performance. Typically, a building block of a deep architecture is Restricted Boltzmann Machine (RBM). In this work, we focus on a variant of RBM adopted to the classification setting, which is known as Classification Restricted Bolt...

We expand the item response theory to study the case of “cheating students” for a set of exams, trying to detect them by applying a greedy algorithm of inference. This extended model is closely related to the Boltzmann machine learning. In this paper we aim to infer the correct biases and interactions of our model by considering a relatively small number of sets of training data. Nevertheless, ...

Boltzmann machines are often used as building blocks in greedy learning of deep networks. However, training even a simplified model, known as restricted Boltzmann machine (RBM), can be extremely laborious: Traditional learning algorithms often converge only with the right choice of the learning rate scheduling and the scale of the initial weights. They are also sensitive to specific data repres...

Deep neural networks have become increasingly more popular under the name of deep learning recently due to their success in challenging machine learning tasks. Although the popularity is mainly due to the recent successes, the history of neural networks goes as far back as 1958 when Rosenblatt presented a perceptron learning algorithm. Since then, various kinds of artificial neural networks hav...

F.X. Albizuri, A.I. Gonzalez, M. Graña, A. d’Anjou University of the Basque Country Informatika Fakultatea, P.K. 649, 20080 Donostia, Spain E-mail: [email protected]; Fax: + 34 943 219306 Abstract. In this paper we define a Boltzmann machine for modelling probability distributions on categorical data, that is, distributions on a set of variables with a finite discrete range. The distribution m...

The measurement of pitch perturbation has been used in acoustic screening for the detection of vocal disorders. As an alternative to applying conventional statistical techniques to the pitch perturbation parameters used for Separation of control and pathological speakers, a parallel distributed processing model approach has been implemented using a Boltzmann Machine (BM). After training, the ma...

We compare the performance of several hidden variable models, namely binary factor analysis, topic models (latent Dirichlet allocation, correlated topic model), the restricted Boltzmann machine and the deep belief net. We shortly present these models and outline their estimation. Performance is measured by log likelihood values of these models for a holdout data set of market baskets. For each ...