A few iterations of alternating least squares with a random starting point provably suffice to produce nearly optimal spectraland Frobenius-norm accuracies of low-rank approximations to a matrix; iterating to convergence is unnecessary. Thus, software implementing alternating least squares can be retrofitted via appropriate setting of parameters to calculate nearly optimally accurate low-rank a...

The low multilinear rank approximation, also known as the truncated Tucker decomposition, has been extensively utilized in many applications that involve higher-order tensors. Popular methods for approximation usually rely directly on matrix SVD, therefore often suffer from notorious intermediate data explosion issue and are not easy to parallelize, especially when input tensor is large. In thi...

Tensor decompositions, such as CANDECOMP/PARAFAC (CP), are widely used in a variety of applications, chemometrics, signal processing, and machine learning. A broadly method for computing decompositions relies on the Alternating Least Squares (ALS) algorithm. When number components is small, regardless its implementation, ALS exhibits low arithmetic intensity, which severely hinders performance ...

In this paper, we apply the alternating direction method (ADM) to solve a constrained linear least-squares problem where the objective function is a sum of two least-squares terms and the constraints are box constraints. Using ADM, we decompose the original problem into two easier least-squares subproblems at each iteration. To speed up the inner iteration, we linearize the subproblems whenever...

Tensors are data structures indexed along three or more dimensions. Tensors have found increasing use in domains such as data mining and recommender systems where dimensions can have enormous length and are resultingly very sparse. The canonical polyadic decomposition (CPD) is a popular tensor factorization for discovering latent features and is most commonly found via the method of alternating...

Least Squares (LS) estimation is a classical problem, often arising in practice. When the dimension of the problem is large, the solution may be difficult to obtain, due to complexity reasons. A general way to reduce the complexity is that of breaking the problem in smaller sub-problems. Following this approach, in the paper we introduce an Alternating Least Squares (ALS) algorithm that finds t...

This work was originally motivated by a classification of tensors proposed by Richard Harshman. In particular, we focus on simple and multiple “bottlenecks”, and on “swamps”. Existing theoretical results are surveyed, some numerical algorithms are described in details, and their numerical complexity is calculated. In particular, the interest in using the ELS enhancement in these algorithms is d...