Stochastic Spectral Descent for Discrete Graphical Models

Stochastic Spectral Descent for Restricted Boltzmann Machines

Restricted Boltzmann Machines (RBMs) are widely used as building blocks for deep learning models. Learning typically proceeds by using stochastic gradient descent, and the gradients are estimated with sampling methods. However, the gradient estimation is a computational bottleneck, so better use of the gradients will speed up the descent algorithm. To this end, we first derive upper bounds on t...

Lecture 21: Spectral Learning for Graphical Models

In modern machine learning, latent variables are often introduced into the models to endow them with learnable and interpretable structures. Examples of such models include various state space models of sequential data (such as hidden Markov models), mixed membership models (such as topic models), and stochastic grammars (such as probabilistic context free grammars) used to model grammatical st...

Efficient Model Determination for Discrete Graphical Models

Spectral Algorithms for Graphical Models Lecturer : Eric

Modern machine learning tasks often deal with high-dimensional data. One typically makes some assumption on structure, like sparsity, to make learning tractable over high-dimensional instances. Another common assumption on structure is that of latent variables in the generative model. In latent variable models, one attempts to perform inference not only on observed variables, but also on unobse...

MCMC model determination for discrete graphical models

In this paper we compare two alternative MCMC samplers for the Bayesian analysis of discrete graphical models; we present both a hierarchical and a nonhierarchical version of them. We Žrst consider the MC3 algorithm by Madigan and York (1995) for which we propose an extension that allows for a hierarchical prior on the cell counts. We then describe a novel methodology based on a reversible jump...