Quasi-Newton methods for numerical optimization exploit quadratic Taylor polynomial models of the objective function. Trust regions are widely used to ensure the global convergence of these methods. Analogously, response surface methods for stochastic optimization exploit linear and quadratic regression models of the objective function. Ridge analysis is widely used to safeguard the optimizatio...

Quasi-Newton algorithms for unconstrained nonlinear minimization generate a sequence of matrices that can be considered as approximations of the objective function second derivatives. This paper gives conditions under which these approximations can be proved to converge globally to the true Hessian matrix, in the case where the Symmetric Rank One update formula is used. The rate of convergence ...

This paper is a continuation of our previous paper [3] were we presented generalizations of the Dennis-Moré theorem to characterize q-superliner convergences of quasi-Newton methods for solving equations and variational inequalities in Banach spaces. Here we prove Dennis-Moré type theorems for inexact quasi-Newton methods applied to variational inequalities in finite dimensions. We first consid...

Several variants of Nonnegative Matrix Factorization (NMF) have been proposed for supervised classification of various objects. Graph regularized NMF (GNMF) incorporates the information on the data geometric structure to the training process, which considerably improves the classification results. However, the multiplicative algorithms used for updating the underlying factors may result in a sl...

This paper reports theoretical and empirical investigations on the use of quasi-Newton methods to minimize the Optimal Bellman Residual (OBR) of zero-sum two-player Markov Games. First, it reveals that state-of-the-art algorithms can be derived by the direct application of Newton’s method to different norms of the OBR. More precisely, when applied to the norm of the OBR, Newton’s method results...

We propose a novel algorithm for learning a geometric combination of Gaussian kernel jointly with a SVM classifier. This problem is the product counterpart of MKL, with restriction to Gaussian kernels. Our algorithm finds a local solution by alternating a Quasi-Newton gradient descent over the kernels and a classical SVM solver over the instances. We show promising results on well known data se...

A derivative-free Quasi-Newton (DFQN) method previously published [J. Greenstadt, Math. Comp., v. 26, 1972, pp. 145—166] has been revised and simplified. The main modification has the effect of keeping all the successive approximants to the Hessian matrix positive-definite. This, coupled with some improvements in the line search, has enhanced the performance of the method considerably. The resu...

In this paper we propose an iterative learning control scheme based on the quasi-Newton method. The iterative learning control is designed to improve the performance of the systems working cyclically. We consider the general type of systems described by continuously diierentiable operator acting in Banach spaces. The suucient conditions for the convergence of quasi-Newton iterative learning alg...

We study how to use the BFGS quasi-Newton matrices to precondition minimization methods for problems where the storage is critical. We give an update formula which generates matrices using information from the last m iterations, where m is any number supplied by the user. The quasi-Newton matrix is updated at every iteration by dropping the oldest information and replacing it by the newest info...