In a recent paper we introduced a trust-region method with variable norms for unconstrained minimization and we proved standard asymptotic convergence results. Here we will show that, with a simple modification with respect to the sufficient descent condition and replacing the trust-region approach with a suitable cubic regularization, the complexity of this method for finding approximate first...

The conjugate gradient method is one of the most popular methods to solve large-scale unconstrained optimization problems since it does not require second derivative, such as Newton’s or approximations. Moreover, can be applied in many fields neural networks, image restoration, etc. Many complicated are proposed these functions two three terms. In this paper, we propose a simple, easy, efficien...

We prove that the set of initial conditions so that gradient descent converges to strict saddle points has (Lebesgue) measure zero, even for non-isolated critical points, answering an open question in [1].

A modified Polak-Ribière-Polyak conjugate gradient algorithm which satisfies both the sufficient descent condition and the conjugacy condition is presented. These properties are independent of the line search. The algorithms use the standard Wolfe line search. Under standard assumptions we show the global convergence of the algorithm. Numerical comparisons with conjugate gradient algorithms usi...