Non-Gaussian Statistical Models and Their Applications

Statistical modeling plays an important role in various research areas. It provides a way to connect the data with the statistics. Based on the statistical properties of the observed data, an appropriate model can be chosen that leads to a promising practical performance. The Gaussian distribution is the most popular and dominant probability distribution used in statistics, since it has an analytically tractable Probability Density Function (PDF) and analysis based on it can be derived in an explicit form. However, various data in real applications have bounded support or semi-bounded support. As the support of the Gaussian distribution is unbounded, such type of data is obviously not Gaussian distributed. Thus we can apply some non-Gaussian distributions, e.g., the beta distribution, the Dirichlet distribution, to model the distribution of this type of data. The choice of a suitable distribution is favorable for modeling efficiency. Furthermore, the practical performance based on the statistical model can also be improved by a better modeling. An essential part in statistical modeling is to estimate the values of the parameters in the distribution or to estimate the distribution of the parameters, if we consider them as random variables. Unlike the Gaussian distribution or the corresponding Gaussian Mixture Model (GMM), a non-Gaussian distribution or a mixture of non-Gaussian distributions does not have an analytically tractable solution, in general. In this dissertation, we study several estimation methods for the non-Gaussian distributions. For the Maximum Likelihood (ML) estimation, a numerical method is utilized to search for the optimal solution in the estimation of Dirichlet Mixture Model (DMM). For the Bayesian analysis, we utilize some approximations to derive an analytically tractable solution to approximate the distribution of the parameters. The Variational Inference (VI) framework based method has been shown to be efficient for approximating the parameter distribution by several researchers. Under this framework, we adapt the conventional Factorized Approximation (FA) method to the Extended Factorized Approximation (EFA) method and use it to approximate the parameter distribution in the beta distribution. Also, the Local Variational Inference (LVI) method is applied to approximate the predictive distribution of the beta distribution. Finally, by assigning a beta distribution to each element in the matrix, we proposed a variational Bayesian Nonnegative Matrix Factorization (NMF) for bounded support data. The performances of the proposed non-Gaussian model based methods are evaluated by several experiments. The beta distribution and the Dirichlet distribution are applied to model the Line Spectral Frequency (LSF) representation of the Linear Prediction (LP) model for statistical model based speech coding. For some image processing applications, the beta distribution is also applied. The proposed beta distribution based variational Bayesian NMF is applied for image restoration and collaborative filtering. Compared to some conventional statistical model based methods, the non-Gaussian model based methods show a promising improvement.