In this paper we give an review on convergence problems of un-constrained optimization algorithms, including line search algorithms and trust region algorithms. Recent results on convergence of conjugate gradient methods are discussed. Some well-known convergence problems of variable metric methods and recent eeorts made on these problems are also presented.

A local minimum is a point at which the value of the function is less than or equal to all immediately nearby or surrounding function values. A global minimum realizes the smallest possible value of the function over all feasible inputs. In general, based purely on local knowledge of a function and its behavior, it can be very difficult to distinguish whether you have discovered a local or a gl...

We consider a family of damped quasi-Newton methods for solving unconstrained optimization problems. This family resembles that of Broyden with line searches, except that the change in gradients is replaced by a certain hybrid vector before updating the current Hessian approximation. This damped technique modifies the Hessian approximations so that they are maintained sufficiently positive defi...

In this paper, we present a nonmonotone trust-region algorithm for unconstrained optimization. We first introduce a variant of the nonmonotone strategy proposed by Ahookhosh and Amini cite{AhA 01} and incorporate it into the trust-region framework to construct a more efficient approach. Our new nonmonotone strategy combines the current function value with the maximum function values in some pri...

Time-series predictions by artificial neural networks (ANNs) are traditionally formulated as unconstrained optimization problems. As an unconstrained formulation provides little guidance on search directions when a search gets stuck in a poor local minimum, we have proposed to use a constrained formulation in order to use constraint violations to provide additional guidance. In this paper, we f...

Time-series predictions by artificial neural networks (ANNs) are traditionally formulated as unconstrained optimization problems. As an unconstrained formulation provides little guidance on search directions when a search gets stuck in a poor local minimum, we have proposed recently to use a constrained formulation in order to use constraint violations to provide additional guidance. In this pa...

This paper deals with the preliminary design problem when the product is modeled as an analytic model. The analytic models based method aims to use mathematical equations to address both multiphysic and economic characteristics of a product. The proposed approach is to convert the preliminary design problem into a global constrained optimization problem. The objective is to develop powerful opt...

This paper reviews advances in Newton quasi Newton and conjugate gradi ent methods for large scale optimization It also describes several packages developed during the last ten years and illustrates their performance on some practical problems Much attention is given to the concept of partial separa bility which is gaining importance with the arrival of automatic di erentiation tools and of opt...