In order to represent state in controlled, partially observable, stochastic dynamical systems, some sort of sufficient statistic for history is necessary. Predictive representations of state (PSRs) capture state as statistics of the future. We introduce a new model of such systems called the “Exponential family PSR,” which defines as state the time-varying parameters of an exponential family di...

We investigate the set of parameters κ ∈ C for which the singular orbit (0, eκ, . . .) of Eκ(z) := exp(z + κ) converges to ∞. These parameters are organized in smooth curves in parameter space called parameter rays.

We consider the matrix completion problem of recovering a structured matrix from noisy and partial measurements. Recent works have proposed tractable estimators with strong statistical guarantees for the case where the underlying matrix is low–rank, and the measurements consist of a subset, either of the exact individual entries, or of the entries perturbed by additive Gaussian noise, which is ...

The success of Support Vector Machine (SVM) gave rise to the development of a new class of theoretically elegant learning machines which use a central concept of kernels and the associated reproducing kernel Hilbert space (RKHS ). Exponential families, a standard tool in statistics, can be used to unify many existing machine learning algorithms based on kernels (such as SVM) and to invent novel...

Curved exponential family models are a useful generalization of exponential random graph models (ERGMs). In particular, models involving the alternating k-star, alternating k-triangle, and alternating k-twopath statistics of Snijders et al (2006) may be viewed as curved exponential family models. This article unifies recent material in the literature regarding curved exponential family models f...

Many applications involve large collections of high-dimensional datapoints with noisy entries from exponential family distributions. It is of interest to estimate the covariance and principal components of the noiseless distribution. In photon-limited imaging (e.g. XFEL) we want to estimate the covariance of the pixel intensities of 2-D images, where the pixels are low-intensity Poisson variabl...

Abstract We present a method for learning max-weight matching predictors in bipartite graphs. The method consists of performing maximum a posteriori estimation in exponential families with sufficient statistics that encode permutations and data features. Although inference is in general hard, we show that for one very relevant application–document ranking–exact inference is efficient. For gener...

For the parameter estimation problem, we know nothing about the parameter but the observations from such a distribution. Therefore, the observations X1, · · · , Xn is our first hand of information source about the parameter, that is to say, all the available information about the parameter is contained in the observations. However, we know that the estimators we obtained are always functions of...

We introduce generalized notions of a divergence function and a Fisher information matrix. We propose to generalize the notion of an exponential family of models by reformulating it in terms of the Fisher information matrix. Our methods are those of information geometry. The context is general enough to include applications from outside statistics.

This paper deals with evolutionary clustering, which refers to the problem of clustering data with distribution drifting along time. Starting from a density estimation view to clustering problems, we propose two general on-line frameworks. In the first framework, i.e., historical data dependent (HDD), current model distribution is designed to approximate both current and historical data distrib...