Novel coordinate descent (CD) methods are proposed for minimizing nonconvex functions consisting of three terms: (i) a continuously differentiable term, (ii) simple convex and (iii) concave continuous term. First, by extending randomized CD to nonsmooth settings, we develop subgradient method that randomly updates block-coordinate variables using block composite mapping. This converges asymptot...

We propose two simple polynomial-time algorithms to find a positive solution to Ax = 0. Both algorithms iterate between coordinate descent steps similar to von Neumann’s algorithm, and rescaling steps. In both cases, either the updating step leads to a substantial decrease in the norm, or we can infer that the condition measure is small and rescale in order to improve the geometry. We also show...

We describe an asynchronous parallel stochastic coordinate descent algorithm for minimizing smooth unconstrained or separably constrained functions. The method achieves a linear convergence rate on functions that satisfy an essential strong convexity property and a sublinear rate (1/K) on general convex functions. Near-linear speedup on a multicore system can be expected if the number of proces...

Algorithms with fast convergence, small number of data access, and low periteration complexity are particularly favorable in the big data era, due to the demand for obtaining highly accurate solutions to problems with a large number of samples in ultra-high dimensional space. Existing algorithms lack at least one of these qualities, and thus are inefficient in handling such big data challenge. ...

We present a unified framework for online learning of structured classifiers. This framework handles a wide family of convex loss functions that includes as particular cases CRFs, structured SVMs, and the structured perceptron. We introduce a new aggressive online algorithm that optimizes any loss in this family; for the structured hinge loss, this algorithm reduces to 1-best MIRA; in general, ...