A spectral algorithm for learning mixture models

A mixture model is a weighted combination of probability distributions. We consider the problem of identifying the component distributions of a mixture model by examining random samples from the mixture. Our main result is that a simple spectral algorithm for learning a mixture of k spherical Gaussians in n-dimensions works remarkably well — it succeeds in identifying the Gaussians assuming essentially the minimum possible separation between their centers that keeps them unique. Unlike existing algorithms, the sample complexity and running time are polynomial in both n and k. We then apply it to the more general problem of learning a mixture of weakly isotropic distributions (e.g. a mixture of uniform distributions on cubes). The algorithm is robust in that it can tolerate small amounts of noise and thus can also be used for the more general problem of finding the best mixture model that fits a given data set, provided there exists a good fit.