where U = V k−mV k−m+1 · · ·V k−1. For the L-BFGS, we need not explicitly store the approximated inverse Hessian matrix. Instead, we only require matrix-vector multiplications at each iteration, which can be implemented by a twoloop recursion with a time complexity of O(mn) (Jorge & Stephen, 1999). Thus, we only store 2m vectors of length n: sk−1, sk−2, · · · , sk−m and yk−1,yk−2, · · · ,yk−m w...

In this paper, we propose a filter active-set algorithm for the minimization problem over a product of multiple ball/sphere constraints. By making effective use of the special structure of the ball/sphere constraints, a new limited memory BFGS (L-BFGS) scheme is presented. The new L-BFGS implementation takes advantage of the sparse structure of the Jacobian of the constraints, and generates cur...

In this paper, we propose a second order optimization method to learn models where both the dimensionality of the parameter space and the number of training samples is high. In our method, we construct on each iteration a Krylov subspace formed by the gradient and an approximation to the Hessian matrix, and then use a subset of the training data samples to optimize over this subspace. As with t...

The limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) optimization method performs very efficiently for large-scale problems. A trust region search generally more and robustly than a line method, especially when the gradient of objective function cannot be accurately evaluated. computational cost an L-BFGS subproblem (TRS) solver depend mainly on number unknown variables ( n ) variable s...

The selection of the hyper-parameters plays a critical role in task prediction based on recurrent neural networks (RNN). Traditionally, machine learning models are selected by simulations as well human experiences. In recent years, multiple algorithms Bayesian optimization (BO) developed to determine optimal values hyper-parameters. most these methods, gradients required be calculated. this wor...

This paper deals with the estimation of optical property distributions of participating media from a set of light sources and sensors located on the boundaries of the medium. This is the so-called diffuse optical tomography problem. Such a non-linear ill-posed inverse problem is solved through the minimization of a cost function which depends on the discrepancy, in a leastsquare sense, between ...

This paper addresses the equal circle packing problem, and proposes an efficient quasi-physical algorithm(EQPA). EQPA is based on an improved BFGS algorithm and a new basin hopping strategy. Starting form a random initial pattern, we use the modified BFGS algorithm to reach a local minimum pattern. The modified BFGS algorithm fully utilizes the neighborhood information and considerably speeds u...

In this paper we consider a number of real world case studies using an automatic design optimisation system called Nimrod/O. The case studies include a photochemical pollution model, two different simulations of the strength of a mechanical part and the radio frequency properties of a ceramic bead. In each case the system is asked to minimise an objective function that results from the executio...