In this paper we suggest another accelerated conjugate gradient algorithm that for all both the descent and the conjugacy conditions are guaranteed. The search direction is selected as where , The coefficients 0 k ≥ 1 1 1 1 ( / ) ( / ) T T T T k k k k k k k k k k k k k k d g y g y s s t s g y s θ + + + + = − + − , s 1 1 ( ) k k g f x + + = ∇ 1 . k k k s x x + = − k θ and in this linear combinat...

In maximizing a non-linear function G(0), it is well known that the steepest descent method has a slow convergence rate. Here we propose a systematic procedure to obtain a 1-1 transformation on the variables 0, so that in the space of the transformed variables, the steepest descent method produces the solution faster. The final solution in the original space is obtained by taking the inverse tr...

Thus we can apply any optimization algorithm to solve this minimization problem and obtain a method for solving (1.1). At the point, let us consider the steepest descent method and select any initial guess x0. With xk available we try to find the direction along which φ(x) decreases most rapidly starting from xk and compute the next point xk+1 by minimizing φ(x) in this direction. By Taylor ser...

The conjugate gradient method is one of the most popular methods to solve large-scale unconstrained optimization problems since it does not require second derivative, such as Newton’s or approximations. Moreover, can be applied in many fields neural networks, image restoration, etc. Many complicated are proposed these functions two three terms. In this paper, we propose a simple, easy, efficien...

In this paper, it is aimed to computationally conduct a performance benchmarking for the steepest descent and three well-known conjugate gradient methods (i.e., Fletcher-Reeves, Polak-Ribiere Hestenes-Stiefel) along with six different step length calculation techniques/conditions, namely Backtracking, Armijo-Backtracking, Goldstein, weakWolfe, strongWolfe, Exact local minimizer in unconstrained...

Many scientiic applications require to solve successively linear systems Ax = b with diierent right-hand sides b and a symmetric positive deenite matrix A. The Conjugate Gradient method applied to the rst system generates a Krylov subspace which can be eeciently recycled thanks to orthogonal projections in subsequent systems. A modiied Conjugate Gradient method is then applied with a speciic in...

Memory gradient methods are used for unconstrained optimization, especially large scale problems. The first idea of memory gradient methods was proposed by Miele and Cantrell (1969) and subsequently extended by Cragg and Levy (1969). Recently Narushima and Yabe (2006) proposed a new memory gradient method which generates a descent search direction for the objective function at every iteration a...

A tuning method was proposed for automatic lighting (auto-lighting) algorithms derived from the steepest descent and conjugate gradient methods. The auto-lighting algorithms maximize the image quality of industrial machine vision by adjusting multiple-color light emitting diodes (LEDs)—usually called color mixers. Searching for the driving condition for achieving maximum sharpness influences im...

Several recent empirical studies demonstrate that important machine learning tasks such as training deep neural networks, exhibit a low-rank structure, where most of the variation in loss function occurs only few directions input space. In this paper, we leverage structure to reduce high computational cost canonical gradient-based methods gradient descent (GD). Our proposed Low-Rank Gradient De...