In this paper the problem of forecasting high dimensional time series is considered. Such time series can be modeled as matrices where each column denotes a measurement. In addition, when missing values are present, low rank matrix factorization approaches are suitable for predicting future values. This paper formally defines and analyzes the forecasting problem in the online setting, i.e. wher...

We present a novel approach to collaborative prediction, using low-norm instead of low-rank factorizations. The approach is inspired by, and has strong connections to, large-margin linear discrimination. We show how to learn low-norm factorizations by solving a semi-definite program, and discuss generalization error bounds for them.

Many kinds of data can be viewed as consisting of a set of vectors, each of which is a noisy combination of a small number of noisy prototype vectors. Physically, these prototype vectors may correspond to different hidden variables that play a role in determining the measured data. For example, a gene’s expression is influenced by the presence of transcription factor proteins, and two genes may...

Large matrices appearing in machine learning problems often have complex hierarchical structures that go beyond what can be found by traditional linear algebra tools, such as eigendecomposition. Inspired by ideas from multiresolution analysis, this paper introduces a new notion of matrix factorization that can capture structure in matrices at multiple different scales. The resulting Multiresolu...

In Nonnegative Matrix Factorization (NMF), a nonnegative matrix is approximated by a product of lower-rank factorizing matrices. Most NMF methods assume that each factorizing matrix appears only once in the approximation, thus the approximation is linear in the factorizing matrices. We present a new class of approximative NMF methods, called Quadratic Nonnegative Matrix Factorization (QNMF), wh...

Matrix factorization (MF) is the simplest and most well studied factor based model and has been applied successfully in several domains. One of the standard ways to solve MF is by finding maximum a posteriori estimate of the model parameters, which is equivalent to minimizing the regularized objective function. Stochastic gradient descent (SGD) is a common choice to minimize the regularized obj...

As a result of recent interdisciplinary work in signal processing (audio, still-images, etc), a number of powerful matrix operations have led to advances both in engineering applications and in mathematics. Much of it is motivated by ideas from wavelet algorithms. The applications are convincing measured against other processing tools already available, for example better compression (details b...

Pietsch Factorization and Grothendieck Factorization are the two landmark theorems in modern functional analysis. They were first introduced to the numerical linear algebra community by the work of Joel A. Tropp in the column subset selection problem, which seeks to extract from a matrix a column submatrix that has lower spectral norm. Despite their broad application in functional analysis, the...

Nonnegative matrix factorization (NMF) is a powerful tool for data mining. However, the emergence of ‘big data’ has severely challenged our ability to compute this fundamental decomposition using deterministic algorithms. This paper presents a randomized hierarchical alternating least squares (HALS) algorithm to compute the NMF. By deriving a smaller matrix from the nonnegative input data, a mo...

Many existing approaches to collaborative filtering can neither handle very large datasets nor easily deal with users who have very few ratings. In this paper we present the Probabilistic Matrix Factorization (PMF) model which scales linearly with the number of observations and, more importantly, performs well on the large, sparse, and very imbalanced Netflix dataset. We further extend the PMF ...