Kernel ridge regression is gaining popularity as a data-rich nonlinear forecasting tool, which is applicable in many different contexts. This paper investigates the influence of the choice of kernel and the setting of tuning parameters on forecast accuracy. We review several popular kernels, including polynomial kernels, the Gaussian kernel, and the Sinc kernel. We interpret the latter two kern...

This paper derives an identity connecting the square loss of ridge regression in on-line mode with the loss of the retrospectively best regressor. Some corollaries about the properties of the cumulative loss of on-line ridge regression are also obtained.

T HE introduction by Hoerl and Kennard (1970) of a ridge regression estimator to deal with the problem of multicollinearity in regression has been followed by a large number of papers in the statistical literature. In the area of econometrics, though, the method of ridge regression has received little attention. I One of the reasons for the lack of interest in ridge regression on the part of th...

In this paper we study a dual version of the Ridge Regression procedure. It allows us to perform non-linear regression by constructing a linear regression function in a high dimensional feature space. The feature space representation can result in a large increase in the number of parameters used by the algorithm. In order to combat this \curse of dimensionality", the algorithm allows the use o...

Technological advances facilitating the acquisition of large arrays of biomarker data have led to new opportunities to understand and characterize disease progression over time. This creates an analytical challenge, however, due to the large numbers of potentially informative markers, the high degrees of correlation among them, and the time-dependent trajectories of association. We propose a mi...

This paper discusses the effect of hubness in zero-shot learning, when ridge regression is used to find a mapping between the example space to the label space. Contrary to the existing approach, which attempts to find a mapping from the example space to the label space, we show that mapping labels into the example space is desirable to suppress the emergence of hubs in the subsequent nearest ne...

Ridge regression is often favored in the analysis of ill-conditioned systems. A canonical form identifies regions in the parameter space where Ordinary Least Squares (OLS) is problematic. The objectives are two-fold: To reexamine the view that ill-conditioning necessarily degrades essentials of OLS; and to reassess ranges of the ridge parameter k where ridge is efficient in mean squared error (...

This work gives a simultaneous analysis of both the ordinary least squares estimator and the ridge regression estimator in the random design setting under mild assumptions on the covariate/response distributions. In particular, the analysis provides sharp results on the “out-of-sample” prediction error, as opposed to the “in-sample” (fixed design) error. The analysis also reveals the effect of ...