Definition A Gaussian Mixture Model (GMM) is a parametric probability density function represented as a weighted sum of Gaussian component densities. GMMs are commonly used as a parametric model of the probability distribution of continuous measurements or features in a biometric system, such as vocal-tract related spectral features in a speaker recognition system. GMM parameters are estimated ...

Consider the problem of tting a nite Gaussian mixture, with an unknown number of components, to observed data. This paper proposes a new minimum description length (MDL) type criterion, termed MMDL (formixtureMDL), to select the number of components of the model. MMDL is based on the identi cation of an \equivalent sample size", for each component, which does not coincide with the full sample s...

We introduce a generative model, we call Tensorial Mixture Models (TMMs) based on mixtures of basic component distributions over local structures (e.g. patches in an image) where the dependencies between the local-structures are represented by a ”priors tensor” holding the prior probabilities of assigning a component distribution to each local-structure. In their general form, TMMs are intracta...

This paper studies sparse density estimation via l1 penalization (SPADES). We focus on estimation in high-dimensional mixture models and nonparametric adaptive density estimation. We show, respectively, that SPADES can recover, with high probability, the unknown components of a mixture of probability densities and that it yields minimax adaptive density estimates. These results are based on a g...

Many currently popular models of categorization are either strictly parametric (e.g., prototype models, decision bound models) or strictly nonparametric (e.g., exemplar models) (Ashby & Alfonso-Reese, 1995). In this article, a family of semi-parametric classifiers is investigated where categories are represented by a finite mixture distribution. The advantage of these mixture models of categori...

The use of Riemannian manifolds and their statistics has recently gained popularity in a wide range of applications involving non-linear data modeling. For instance, they have been used to model shape changes in the brain [1] and human motion [3]. In this work we tackle the problem of approximating the Probability Density Function (PDF) of a potentially large dataset that lies on a known Rieman...

This paper introduces two data partitioning methods for building mixtures of several neural networks. The methods are based on active learning with two different selection measures. One is the redundant data selection (RDS) method whiCh chooses examples with less error, and the other is the critical data selection (CDS) method which chooses examples with larger error. The partitioned data sets ...

doi: 10.3969/j.issn.1002-0829.2012.06.009 Division of Biostatistics, Thomas Jefferson University, Philadelphia, Pennsylvania, USA *Correspondence: [email protected] Psychiatric studies often collect longitudinal data to characterize the natural history of disease in a cohort or to evaluate the effect of behavioral or pharmaceutical interventions. For example, in a recent partially ra...

Factor analysis is a statistical method for describing the associations among sets of observed variables in terms of a small number of underlying continuous latent variables. Various authors have proposed multilevel extensions of the factor model for the analysis of data sets with a hierarchical structure. These Multilevel Factor Models (MFMs) have in common that-as in multilevel regression ana...

Recent neural network sequence models with softmax classifiers have achieved their best language modeling performance only with very large hidden states and large vocabularies. Even then they struggle to predict rare or unseen words even if the context makes the prediction unambiguous. We introduce the pointer sentinel mixture architecture for neural sequence models which has the ability to eit...