A Hessian matrix in full waveform inversion (FWI) is difficult to compute directly because of high computational cost and an especially large memory requirement. Therefore, Newton-like methods are rarely feasible in realistic large-size FWI problems. We modify the quasi-Newton BFGS method to use a projected Hessian matrix that reduces both the computational cost and memory required, thereby mak...

We propose a generic method for iteratively approximating various second-order gradient steps - Newton, Gauss-Newton, Levenberg-Marquardt, and natural gradient - in linear time per iteration, using special curvature matrix-vector products that can be computed in O(n). Two recent acceleration techniques for on-line learning, matrix momentum and stochastic meta-descent (SMD), implement this appro...

Quasi-Newton methods have played a prominent role, over many years, in the design of effective practical methods for the numerical solution of nonlinear minimization problems and in multi-dimensional zero-finding. There is a wide literature outlining the properties of these methods and illustrating their performance [e.g., [8]]. In addition, most modern optimization libraries house a quasi-Newt...

Full Waveform Inversion (FWI) methods use generally gradient based method, such as the nonlinear conjugate gradient method or more recently the l-BFGS quasi-Newton method. Several authors have already investigated the possibility of accounting more accurately for the inverse Hessian operator in the minimization scheme through Gauss-Newton or exact Newton algorithms. We propose a general framewo...

Bootstrap testing of nonlinear models normally requires at least one nonlinear estimation for every bootstrap sample. We show how to reduce computational costs by performing only a fixed, small number of Newton or quasi-Newton steps for each bootstrap sample. The number of steps is smaller for likelihood ratio tests than for other types of classical tests, and smaller for Newton’s method than f...

We present a phase-based potential ordering that is an extension of the Cascade ordering introduced by Appleyard and Cheshire [1]. The ordering is valid for both two-phase and three-phase flow, and it can handle countercurrent flow due to gravity and/or capillarity. We show how this ordering can be used to reduce the nonlinear algebraic system that arise from the fully-implicit method (FIM) int...

Many state-of-the-art approaches to trajectory optimization and optimal control are intimately related to standard Newton methods. For researchers that work in the intersections of machine learning, robotics, control, and optimization, such relations are highly relevant but sometimes hard to see across disciplines, due also to the different notations and conventions used in the disciplines. The...

In this paper, we import interval method to the iteration for computing Moore-Penrose inverse of the full row (or column) rank matrix. Through modifying the classical Newton iteration by interval method, we can get better numerical results. The convergence of the interval iteration is proven. We also give some numerical examples to compare interval iteration with classical Newton iteration.