Solving linear systems is often the computational bottleneck in real-life problems. Iterative solvers are only option due to complexity of direct algorithms or because system matrix not explicitly known. Here, we develop a two-level preconditioner for regularized least squares involving feature data matrix. Variants this may appear machine learning applications, such as ridge regression, logist...

We introduce a proximal version of the stochastic dual coordinate ascent method and show how to accelerate the method using an inner-outer iteration procedure. We analyze the runtime of the framework and obtain rates that improve state-of-the-art results for various key machine learning optimization problems including SVM, logistic regression, ridge regression, Lasso, and multiclass SVM. Experi...

We study the problem of online regression. We do not make any assumptions about input vectors or outcomes. We prove a theoretical bound on the square loss of Ridge Regression. We also show that Bayesian Ridge Regression can be thought of as an online algorithm competing with all the Gaussian linear experts. We then consider the case of infinite-dimensional Hilbert spaces and prove relative loss...

The methods of two-parameter ridge and ordinary regression are very sensitive to the presence joint problem multicollinearity outliers in y-direction. To overcome this problem, modified robust M-estimators proposed. new estimators then compared with existing ones by means extensive Monte Carlo simulations. According mean squared error (MSE) criterion, outperform least square estimator, estimato...