Inexact Variable Metric Stochastic Block-Coordinate Descent for Regularized Optimization
نویسندگان
چکیده
منابع مشابه
Unifying abstract inexact convergence theorems for descent methods and block coordinate variable metric iPiano
An abstract convergence theorem for a class of descent method that explicitly models relative errors is proved. The convergence theorem generalizes and unifies several recent abstract convergence theorems, and is applicable to possibly non-smooth and non-convex lower semi-continuous functions that satisfy the Kurdyka– Lojasiewicz inequality, which comprises a huge class of problems. The descent...
متن کاملRandomized Block Coordinate Descent for Online and Stochastic Optimization
Two types of low cost-per-iteration gradient descent methods have been extensively studied in parallel. One is online or stochastic gradient descent ( OGD/SGD), and the other is randomzied coordinate descent (RBCD). In this paper, we combine the two types of methods together and propose online randomized block coordinate descent (ORBCD). At each iteration, ORBCD only computes the partial gradie...
متن کاملAccelerated Block-coordinate Relaxation for Regularized Optimization
We discuss minimization of a smooth function to which is added a regularization function that induces structure in the solution. A block-coordinate relaxation approach with proximal linearized subproblems yields convergence to stationary points, while identification of the optimal manifold (under a nondegeneracy condition) allows acceleration techniques to be applied on a reduced space. The wor...
متن کاملStochastic Coordinate Descent for Nonsmooth Convex Optimization
Stochastic coordinate descent, due to its practicality and efficiency, is increasingly popular in machine learning and signal processing communities as it has proven successful in several large-scale optimization problems , such as l1 regularized regression, Support Vector Machine, to name a few. In this paper, we consider a composite problem where the nonsmoothness has a general structure that...
متن کاملStochastic Coordinate Descent Methods for Regularized Smooth and Nonsmooth Losses
Stochastic Coordinate Descent (SCD) methods are among the first optimization schemes suggested for efficiently solving large scale problems. However, until now, there exists a gap between the convergence rate analysis and practical SCD algorithms for general smooth losses and there is no primal SCD algorithm for nonsmooth losses. In this paper, we discuss these issues using the recently develop...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Optimization Theory and Applications
سال: 2020
ISSN: 0022-3239,1573-2878
DOI: 10.1007/s10957-020-01639-4