Many canonical machine learning problems boil down to a convex optimization problem with a finite sum structure. However, whereas much progress has been made in developing faster algorithms for this setting, the inherent limitations of these problems are not satisfactorily addressed by existing lower bounds. Indeed, current bounds focus on first-order optimization algorithms, and only apply in ...

In this paper we study the language of the words that, for a given language L, distinguish between pairs of different left-quotients of L. We characterize this distinguishability operation, show that its iteration has always a fixed point, and we generalize this result to operations derived from closure operators and Boolean operators. We give an upper bound for the state complexity of the dist...

Based on a new kernel function, a large-update primal-dual interior-point algorithm for solving linear optimization is proposed. The kernel function is used both for determining the search directions and for measuring the distance between the given iterate and the μ-center for the algorithm. By using several new technical lemmas, the iteration complexity bound as O( √ n log n log nε ) is obtain...

Roos proved that the devised full-step infeasible algorithm has O(n) worst-case iteration complexity. This complexity bound depends linearly on a parameter ¯ κ(ζ), which is proved to be less than √ 2n. Based on extensive computational evidence (hundreds of thousands of randomly generated problems), Roos conjectured that ¯ κ(ζ) = 1 (Conjecture 5.1 in the above-mentioned paper), which would yield...

In this paper we study the language of the words that, for a given language L, distinguish between pairs of different left-quotients of L. We characterize this distinguishability operation, show that its iteration has always a fixed point, and we generalize this result to operations derived from closure operators and Boolean operators. We give an upper bound for the state complexity of the dist...

A predictor-corrector method for solving linear programs from infeasible starting points is analyzed. The method is quadratically convergent and can be combined with Ye's finite termination scheme under very general assumptions. If the starting points are large enough then the algorithm has O(nL) iteration complexity. If the ratio between feasibility and optimality at the starting points is sma...

We study the problem of estimating low-rank matrices from linear measurements (a.k.a., matrix sensing) through nonconvex optimization. We propose an efficient stochastic variance reduced gradient descent algorithm to solve a nonconvex optimization problem of matrix sensing. Our algorithm is applicable to both noisy and noiseless settings. In the case with noisy observations, we prove that our a...

1 College of Communication Engineering, Jilin University, Changchun 130012, China ([email protected]) 2 School of Electronics and Information Engineering, Shenyang Aerospace University, Shenyang 110136, China Abstract A subspace based speech enhancement algorithm with low complexity is proposed by using our improved subspace iteration method. Due to the use of the iteration, the proposed algorithm ...

A low complexity iterative receiver structure is proposed for a convolutionally coded system in a severe ISI environment where diversity reception is available. The receiver performs two successive soft decisions, £rst from a MMSE £lter and then from a channel decoder. At each iteration, extrinsic information is extracted from the detection and decoding stages and is used as a priori informatio...

This paper presents and studies the iteration-complexity of two new inexact variants of Rockafellar’s proximal method of multipliers (PMM) for solving convex programming (CP) problems with a finite number of functional inequality constraints. In contrast to the first variant which solves convex quadratic programming (QP) subproblems at every iteration, the second one solves convex constrained q...