The sparse principal component analysis is a variant of the classical principal component analysis, which finds linear combinations of a small number of features that maximize variance across data. In this paper we propose a methodology for adding two general types of feature grouping constraints into the original sparse PCA optimization procedure. We derive convex relaxations of the considered...

In this simple note, we attempt to further improve the sparse principal component analysis (SPCA) of Zou et al. (2006) on the following two aspects. First, we replace the traditional lasso penalty utilized in the original SPCA by the most recently developed adaptive lasso penalty (Zou, 2006; Wang et al., 2006). By doing so, adaptive amounts of shrinkage can be applied to different loading coeff...

In this article, we propose a new method for principal component analysis (PCA), whose main objective is to capture natural "blocking" structures in the variables. Further, the method, beyond selecting different variables for different components, also encourages the loadings of highly correlated variables to have the same magnitude. These two features often help in interpreting the principal c...

We produce approximation bounds on a semidefinite programming relaxation for sparse principal component analysis. These bounds control approximation ratios for tractable statistics in hypothesis testing problems where data points are sampled from Gaussian models with a single sparse leading component. We study approximation bounds for a semidefinite relaxation of the sparse eigenvalue problem, ...

Principal component analysis (PCA) is a popular dimension reduction method that approximates a numerical data matrix by seeking principal components (PC), i.e. linear combinations of variables that captures maximal variance. Since each PC is a linear combination of all variables of a data set, interpretation of the PCs can be difficult, especially in high-dimensional data. In order to find ’spa...

The method of sparse principal component analysis (S-PCA) proposed by Zou, Hastie, and Tibshirani (2006) is an attractive approach to obtain sparse loadings in principal component analysis (PCA). S-PCA was motivated by reformulating PCA as a least-squares problem so that a lasso penalty on the loading coefficients can be applied. In this article, we propose new estimates to improve S-PCA in the...

We introduce a new method for sparse principal component analysis, based on the aggregation of eigenvector information from carefully-selected random projections of the sample covariance matrix. Unlike most alternative approaches, our algorithm is non-iterative, so is not vulnerable to a bad choice of initialisation. Our theory provides great detail on the statistical and computational trade-of...

We consider analysis of sparsely sampled multilevel functional data, where the basic observational unit is a function and data have a natural hierarchy of basic units. An example is when functions are recorded at multiple visits for each subject. Multilevel functional principal component analysis (MFPCA; Di et al. 2009) was proposed for such data when functions are densely recorded. Here we con...

We present an algorithm that computes exactly (optimally) the S-sparse (1≤S<D) maximum-L1-norm-projection principal component of a real-valued data matrix X ∈ RD×N that contains N samples of dimension D. For fixed sample support N , the optimal L1-sparse algorithm has linear complexity in data dimension, O (D). For fixed dimension D (thus, fixed sparsity S), the optimal L1-sparse algorithm has ...

Sparse Principal Component Analysis: Algorithms and Applications