Due to the rapid advance of general-purpose graphics processing unit (GPU), it is an active research topic to study performance improvement of non-linear optimization with parallel implementation on GPU, as attested by the much research on parallel implementation of relatively simple optimization methods, such as the conjugate gradient method. We study in this context the L-BFGS-B method, or th...

A descent algorithm, “Quasi-Quadratic Minimization with Memory” (QQMM), is proposed for unconstrained minimization of the sum, F , of a non-negative convex function, V , and a quadratic form. Such problems come up in regularized estimation in machine learning and statistics. In addition to values of F , QQMM requires the (sub)gradient of V . Two features of QQMM help keep low the number of eval...

Full waveform inversion (FWI) is a non-linear optimization problem based on full-wavefield modeling to obtain quantitative information of subsurface structure by minimizing the difference between observed seismic data and predicted wavefield. The limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) method an effective quasi-Newton in FWI due its high efficiency with low calculation storage ...

We compare the performance of several robust large-scale minimization algorithms for the unconstrained minimization of an ill-posed inverse problem. The parabolized Navier–Stokes equation model was used for adjoint parameter estimation. The methods compared consist of three versions of nonlinear conjugate-gradient (CG) method, quasiNewton Broyden–Fletcher–Goldfarb–Shanno (BFGS), the limited-mem...

In this project I describe the status of inverse kinematics research, with the focus firmly on the methods that solve the core problem. An overview of the different methods are presented Three common methods used in inverse kinematics computation have been chosen as subject for closer inspection. The three methods are described in some detail. An analysis is performed where the three methods ar...

We present a system that enables rapid model experimentation for tera-scale machine learning with trillions of non-zero features, billions of training examples, and millions of parameters. Our contribution to the literature is a new method (SA L-BFGS) for changing batch L-BFGS to perform in near real-time by using statistical tools to balance the contributions of previous weights, old training ...

We consider the problem of the global convergence of gradient-based optimization algorithms for interstitial high-dose-rate (HDR) brachytherapy dose optimization using variance-based objectives. Possible local minima could lead to only sub-optimal solutions. We perform a configuration space analysis using a representative set of the entire non-dominated solution space. A set of three prostate i...

L BFGS B is a limited memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables It is intended for problems in which information on the Hessian matrix is di cult to obtain or for large dense problems L BFGS B can also be used for unconstrained problems and in this case performs similarly to its predecessor algorithm L BFGS Harwell routine VA Th...

A BFGS method, in association with a new backtracking line search technique, is presented for solving symmetric nonlinear equations. The global and superlinear convergences of the given method are established under mild conditions. Preliminary numerical results show that the proposed method is better than the normal technique for the given problems. c © 2007 Elsevier Ltd. All rights reserved.