This paper analyzes and improves the linearized Bregman method for solving the basis pursuit and related sparse optimization problems. The analysis shows that the linearized Bregman method has the exact regularization property; namely, it converges to an exact solution of the basis pursuit problem whenever its smooth parameter α is greater than a certain value. The analysis is based on showing ...

This thesis develops new quasi-Newton optimization methods that exploit the wellstructured functional form of objective functions often encountered in machine learning, while still maintaining the solid foundation of the standard BFGS quasi-Newton method. In particular, our algorithms are tailored for two categories of machine learning problems: (1) regularized risk minimization problems with c...

We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results for learning from expert advice.

We derive compact representations of BFGS and symmetric rank one matrices for optimization These representations allow us to e ciently implement limitedmemory methods for large constrained optimization problems In particular we discuss how to compute projections of limited memory matrices onto subspaces We also present a compact representation of the matrices generated by Broyden s update for s...

On the platform of general chemical process simulation software (it was named Optimization Engineer, OPEN), a optimization algorithm for is developed using C + code. The based on sequential quadratic programming (SQP). We adopt activity set and rotation axis to generate solve sub-problem. active method can simplify number constraints speed up calculation. At same time, we used limited memory BF...

Full waveform inversion (FWI) is an effective and efficient data-fitting technique that has been widely used to produce accurate estimation of model parameters in Geophysics. The efficiency and accuracy of FWI are determined by the three main components: numerical solution for forward problem, gradient calculation and model update which usually involves the optimization method. The success of t...

Phase diversity phase retrieval (PDPR) has been a popular technique for quantitatively measuring wavefront errors of optical imaging systems by extracting the information from several designated intensity measurements. As problem is inverse and non-convex in general, accuracy robustness most such algorithms rely greatly on initial conditions. In this work, we propose new strategy that combines ...

We provide a variable metric stochastic approximation theory. In doing so, we provide a convergence theory for a large class of online variable metric methods including the recently introduced online versions of the BFGS algorithm and its limited-memory LBFGS variant. We also discuss the implications of our results in the areas of eliciting properties of distributions using prediction markets a...

Full waveform inversion (FWI) is an effective and efficient data-fitting technique that has been widely used to produce accurate estimation of model parameters in Geophysics. The efficiency and accuracy of FWI are determined by the three main components: numerical solution for forward problem, gradient calculation and model update which usually involves the optimization method. The success of t...