Metric learning has been shown to significantly improve the accuracy of k-nearest neighbor (kNN) classification. In problems involving thousands of features, distance learning algorithms cannot be used due to overfitting and high computational complexity. In such cases, previous work has relied on a two-step solution: first apply dimensionality reduction methods to the data, and then learn a me...

I introduce Forecastable Component Analysis (ForeCA), a novel dimension reduction technique for temporally dependent signals. Based on a new forecastability measure, ForeCA finds an optimal transformation to separate a multivariate time series into a forecastable and an orthogonal white noise space. I present a converging algorithm with a fast eigenvector solution. Applications to financial and...

I present a method for analyzing multichannel recordings in response to repeated stimulus presentation. Quadratic Component Analysis (QCA) extracts responses that are stimulus-induced (triggered by the stimulus but not precisely locked in time), as opposed to stimulus-evoked (time-locked to the stimulus). Induced responses are often found in neural response data from magnetoencephalography (MEG...

We consider a problem involving estimation of a high-dimensional covariance matrix that is the sum of a diagonal matrix and a low-rank matrix, and making a decision based on the resulting estimate. Such problems arise, for example, in portfolio management, where a common approach employs principal component analysis (PCA) to estimate factors used in constructing the low-rank term of the covaria...

Given a set of signals, a classical construction of an optimal truncatable basis for optimally representing the signals, is the principal component analysis (PCA for short) approach. When the information about the signals one would like to represent is a more general property, like smoothness, a different basis should be considered. One example is the Fourier basis which is optimal for represen...

We propose a feature extraction algorithm, based on the Hilbert–Schmidt independence criterion (HSIC) and the maximum dependence – minimum redundancy approach. Experiments with classification data sets demonstrate that suggested Hilbert–Schmidt component analysis (HSCA) algorithm in certain cases may be more efficient than other considered approaches.

Independent Component Analysis (ICA) (Comon, 1994) was originally proposed to solve the blind source separation problem of recovering independent source signals (e.g., different voice, music, or noise sources) after they are linearly mixed by an unknown matrix, A (cf. Figure 1). Nothing is known about the sources or the mixing process except that there are N different recorded mixtures. The tas...

We consider the online version of the well known Principal Component Analysis (PCA) problem. In standard PCA, the input to the problem is a set of vectors X = [x1, . . . , xn] in Rd×n and a target dimension k < d; the output is a set of vectors Y = [y1, . . . , yn] in Rk×n that minimize minΦ ‖X − ΦY ‖F where Φ is restricted to be an isometry. The global minimum of this quantity, OPTk, is obtain...

Efficient and compact representation of images is a fundamental problem in computer vision. Principal Component Analysis (PCA) has been widely used for image representation and has been successfully applied to many computer vision algorithms. In this paper, we propose a method that uses Haar-like binary box functions to span a subspace which approximates the PCA subspace. The proposed method ca...

In ordinary independent component analysis, the components are assumed to be completely independent, and they do not necessarily have any meaningful order relationships. In practice, however, the estimated "independent" components are often not at all independent. We propose that this residual dependence structure could be used to define a topographic order for the components. In particular, a ...