We extend the well-known BFGS quasi-Newton method and its limited-memory variant (LBFGS) to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: The local quadratic model, the identification of a descent direction, and the Wolfe line search conditions. We apply the resulting subLBFGS algorithm to L2-reg...

Full waveform inversion (FWI) is an effective and efficient data-fitting technique that has been widely used to produce accurate estimation of model parameters in Geophysics. The efficiency and accuracy of FWI are determined by the three main components: numerical solution for forward problem, gradient calculation and model update which usually involves the optimization method. The success of t...

This thesis develops new quasi-Newton optimization methods that exploit the wellstructured functional form of objective functions often encountered in machine learning, while still maintaining the solid foundation of the standard BFGS quasi-Newton method. In particular, our algorithms are tailored for two categories of machine learning problems: (1) regularized risk minimization problems with c...

In this paper, we introduce a new variant of the BFGS method designed to perform well when gradient measurements are corrupted by noise. We show that treating secant condition with penalty approach motivated regularized least squares estimation generates parametric family original update at one extreme and not updating inverse Hessian approximation other extreme. Furthermore, find curvature is ...

"In this paper, we present some hybrid methods for solving unconstrained optimization problems. These are defined using proper combinations of the search directions and included parameters in conjugate gradient quasi-Newton method Broyden-Fletcher-Goldfarb-Shanno (CG-BFGS). Their global convergence under Wolfe line is analyzed general objective functions. Numerical experiments show superiority ...

Full waveform inversion (FWI) is an effective and efficient data-fitting technique that has been widely used to produce accurate estimation of model parameters in Geophysics. The efficiency and accuracy of FWI are determined by the three main components: numerical solution for forward problem, gradient calculation and model update which usually involves the optimization method. The success of t...

The current work focuses on the development and application of an efficient algorithm for optimization of threedimensional turbulent flows, simulated using Direct Numerical Simulation (DNS) or large-eddy simulations, and further characterized by large-dimensional optimization-parameter spaces. The optimization algorithm is based on Sequential Quadratic Programming (SQP) in combination with a da...