Manifold Sampling for L1 Nonconvex Optimization

Manifold Sampling for ℓ1 Nonconvex Optimization

We present a new algorithm, called manifold sampling, for the unconstrained minimization of a nonsmooth composite function h ◦ F when h has known structure. In particular, by classifying points in the domain of the nonsmooth function h into manifolds, we adapt search directions within a trust-region framework based on knowledge of manifolds intersecting the current trust region. We motivate thi...

Manifold Sampling for Nonconvex Optimization of Piecewise Linear Compositions

We develop a manifold sampling algorithm for the unconstrained minimization of 4 a nonsmooth composite function f , ψ + h ◦ F when ψ is smooth with known derivatives, h is a 5 nonsmooth, piecewise linear function, and F is smooth but expensive to evaluate. The trust-region 6 algorithm classifies points in the domain of h as belonging to different manifolds and uses this knowl7 edge when computi...

An Efficient Neurodynamic Scheme for Solving a Class of Nonconvex Nonlinear Optimization Problems

‎By p-power (or partial p-power) transformation‎, ‎the Lagrangian function in nonconvex optimization problem becomes locally convex‎. ‎In this paper‎, ‎we present a neural network based on an NCP function for solving the nonconvex optimization problem‎. An important feature of this neural network is the one-to-one correspondence between its equilibria and KKT points of the nonconvex optimizatio...

An effective optimization algorithm for locally nonconvex Lipschitz functions based on mollifier subgradients

Geometric Optimization in Machine Learning

Machine learning models often rely on sparsity, low-rank, orthogonality, correlation, or graphical structure. The structure of interest in this chapter is geometric, specifically the manifold of positive definite (PD) matrices. Though these matrices recur throughout the applied sciences, our focus is on more recent developments in machine learning and optimization. In particular, we study (i) m...