Solving linear systems is often the computational bottleneck in real-life problems. Iterative solvers are only option due to complexity of direct algorithms or because system matrix not explicitly known. Here, we develop a two-level preconditioner for regularized least squares involving feature data matrix. Variants this may appear machine learning applications, such as ridge regression, logist...

We study the problem of online regression. We do not make any assumptions about input vectors or outcomes. We prove a theoretical bound on the square loss of Ridge Regression. We also show that Bayesian Ridge Regression can be thought of as an online algorithm competing with all the Gaussian linear experts. We then consider the case of infinite-dimensional Hilbert spaces and prove relative loss...

Anomalies persist in the foundations of ridge regression as set forth in Hoerl and Kennard (1970) and subsequently. Conventional ridge estimators and their properties do not follow on constraining lengths of solution vectors using LaGrange’s method, as claimed. Estimators so constrained have singular distributions; the proposed solutions are not necessarily minimizing; and heretofore undiscover...

When independent variables have high linear correlation in a multiple linear regression model, we can have wrong analysis. It happens if we do the multiple linear regression analysis based on common Ordinary Least Squares (OLS) method. In this situation, we are suggested to use ridge regression estimator. We conduct some simulation study to compare the performance of ridge regression estimator ...

The methods of two-parameter ridge and ordinary regression are very sensitive to the presence joint problem multicollinearity outliers in y-direction. To overcome this problem, modified robust M-estimators proposed. new estimators then compared with existing ones by means extensive Monte Carlo simulations. According mean squared error (MSE) criterion, outperform least square estimator, estimato...