Quasi-Newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives. This paper gives conditions under which these approximations can be proved to converge globally to the true Hessian matrix, in the case where the Symmetric Rank One update formula is used. The rate of convergence ...

We study how to use the BFGS quasi-Newton matrices to precondition minimization methods for problems where the storage is critical. We give an update formula which generates matrices using information from the last m iterations, where m is any number supplied by the user. The quasi-Newton matrix is updated at every iteration by dropping the oldest information and replacing it by the newest info...

We compare the performance of several robust large-scale minimization algorithms applied for the minimization of the cost functional in the solution of ill-posed inverse problems related to parameter estimation applied to the parabolized Navier-Stokes equations. The methods compared consist of the conjugate gradient method (CG), Quasi-Newton (BFGS), the limited memory Quasi-Newton (L-BFGS) [1],...

For Quasi-Newton methods, one of the most important challenges is to find an estimate Jacobian matrix as close possible real matrix. While in root-finding problems multi-secant methods are regularly used, optimization, it symmetric (in particular BFGS) that popular. Combining and single update formula would combine their benefits. However, can be proved symmetry property generally not compatibl...

In this paper, we propose a fast one-parameter variant of preconditioned Uzawa (OVPU) method with a scaled preconditioner for solving the saddle point problems, and then we present its convergence region and a formula for finding its optimal parameter. Also performance of the proposed OVPU method with a scaled preconditioner is compared with the existing one or two parameter iterative methods w...

The Broyden class of quasi-Newton updates for inverse Hessian approximation are transformed to the formal BFGS update, which makes possible to generalize the well-known Nocedal method based on the Strang recurrences to the scaled limited-memory Broyden family, using the same number of stored vectors as for the limited-memory BFGS method. Two variants are given, the simpler of them does not requ...

Augmented Lagrangian methods for large-scale optimization usually require efficient algorithms for minimization with box constraints. On the other hand, active-set box-constraint methods employ unconstrained optimization algorithms for minimization inside the faces of the box. Several approaches may be employed for computing internal search directions in the large-scale case. In this paper a mi...

An expectation maximization (EM) algorithm is derived to estimate the parameters of a phylogenetic model, a probabilistic model of molecular evolution that considers the phylogeny, or evolutionary tree, by which a set of present-day organisms are related. The EM algorithm is then extended for use with a combined phylogenetic and hidden Markov model. An efficient method is also shown for computi...

We compare the performance of several robust large-scale minimization algorithms for the unconstrained minimization of an ill-posed inverse problem. The parabolized Navier-Stokes equations model was used for adjoint parameter estimation. The methods compared consist of two versions of the nonlinear conjugate gradient method (CG), Quasi-Newton (BFGS), the limited memory Quasi-Newton (L-BFGS) [15...

Because of its simplicity, low memory requirement, computational cost, and global convergence properties, the Conjugate Gradient (CG) method is most popular iterative mathematical technique for optimizing both linear nonlinear systems. Some classical CG methods, however, have drawbacks such as poor numerical performance in terms iterations function evaluations. To address these shortcomings, re...