This paper is concerned with solving nonconvex learning problems with folded concave penalty. Despite that their global solutions entail desirable statistical properties, there lack optimization techniques that guarantee global optimality in a general setting. In this paper, we show that a class of nonconvex learning problems are equivalent to general quadratic programs. This equivalence facili...

In this paper, we design a numerical algorithm for solving a simple bilevel program where the lower level program is a nonconvex minimization problem with a convex set constraint. We propose to solve a combined problem where the first order condition and the value function are both present in the constraints. Since the value function is in general nonsmooth, the combined problem is in general a...

This paper considers distributed nonconvex optimization with the cost functions being over agents. Noting that information compression is a key tool to reduce heavy communication load for algorithms as agents iteratively communicate neighbors, we propose three primal–dual compressed communication. The first two are applicable general class of compressors bounded relative error and third algorit...

Gradient descent (GD) and stochastic gradient (SGD) are the workhorses of large-scale machine learning. While classical theory focused on analyzing performance these methods in convex optimization problems, most notable successes learning have involved nonconvex optimization, a gap has arisen between practice. Indeed, traditional analyses GD SGD show that both algorithms converge to stationary ...

Motivated by the recent developments of nonconvex penalties in sparsity modeling, we propose a nonconvex optimization model for handing the low-rank matrix recovery problem. Different from the famous robust principal component analysis (RPCA), we suggest recovering low-rank and sparse matrices via a nonconvex loss function and a nonconvex penalty. The advantage of the nonconvex approach lies in...

A Mathematical Program with Linear Complementarity Constraints (MPLCC) is an optimization problem where a continuously differentiable function is minimized on a set defined by linear constraints and complementarity conditions on pairs of complementary variables. This problem finds many applications in several areas of science, engineering and economics and is also an important tool for the solu...

Duality is an important notion for constrained optimization which provides a theoretical foundation for a number of constraint decomposition schemes such as separable programming and for deriving lower bounds in space decomposition algorithms such as branch and bound. However, the conventional duality theory has the fundamental limit that it leads to duality gaps for nonconvex optimization prob...