Mixture of Probabilistic Principal Component Analyzers (MPPCA) is a seminal work in Machine Learning in that it was the first to use PCA to perform clustering and local dimensionality reduction. MPPCA is based upon the mixture of Factor Analyzers (MFA) which is similar to MPPCA except is uses Factor Analysis to estimate the covariance matrix. This algorithm is of interest to me because it is re...

Finding an unconstrained and statistically interpretable reparameterization of a covariance matrix is still an open problem in statistics. Its solution is of central importance in covariance estimation, particularly in the recent high-dimensional data environment where enforcing the positive-definiteness constraint could be computationally expensive. We provide a survey of the progress made in ...

We suggest a method for estimating a covariance matrix on the basis of a sample of vectors drawn from a multivariate normal distribution. In particular, we penalize the likelihood with a lasso penalty on the entries of the covariance matrix. This penalty plays two important roles: it reduces the effective number of parameters, which is important even when the dimension of the vectors is smaller...

Many statistical applications require an estimate of a covariance matrix and/or its inverse. Whenthe matrix dimension is large compared to the sample size, which happens frequently, the samplecovariance matrix is known to perform poorly and may suffer from ill-conditioning. There alreadyexists an extensive literature concerning improved estimators in such situations. In the absence offurther kn...

Estimation of an unstructured covariance matrix is difficult because of its positivedefiniteness constraint. This obstacle is removed by regressing each variable on its predecessors, so that estimation of a covariance matrix is shown to be equivalent to that of estimating a sequence of varying-coefficient and varying-order regression models. Our framework is similar to the use of increasing-ord...

A suitable design of state estimators requires a representative model for capturing the plant behavior and knowledge about the noise statistics, which are generally not known in practical applications. While the measurement noise covariance can be directly derived from the measurement device reproducibility, the choice of the process noise covariance is much less straightforward. Further, proce...

The ensemble Kalman filter (EnKF) is a widely used ensemble-based assimilation method, which estimates the forecast error covariance matrix using a Monte Carlo approach that involves an ensemble of short-term forecasts. While the accuracy of the forecast error covariance matrix is crucial for achieving accurate forecasts, the estimate given by the EnKF needs to be improved using inflation techn...

Missing data in longitudinal studies can create enormous challenges in data analysis when coupled with the positive-definiteness constraint on a covariance matrix. For complete balanced data, the Cholesky decomposition of a covariance matrix makes it possible to remove the positive-definiteness constraint and use a generalized linear model setup to jointly model the mean and covariance using co...

We consider a problem involving estimation of a high-dimensional covariance matrix that is the sum of a diagonal matrix and a low-rank matrix, and making a decision based on the resulting estimate. Such problems arise, for example, in portfolio management, where a common approach employs principal component analysis (PCA) to estimate factors used in constructing the low-rank term of the covaria...

In the analysis of spatial data, the inverse of the covariance matrix needs to be calculated. For example, the inverse is needed for best linear unbiased prediction or kriging, and is repeatedly calculated in the maximum likelihood estimation or the Bayesian inferences. Since the spatial sample size can be quite large, operations on the large covariance matrix can be a numerical challenge if no...