Conjugate gradient methods are an important class of methods for unconstrained optimization, especially for large-scale problems. Recently, they have been much studied. This paper proposes a three-parameter family of hybrid conjugate gradient methods. Two important features of the family are that (i) it can avoid the propensity of small steps, namely, if a small step is generated away from the ...

The performance of CGLS, a basic iterative method whose main idea is to organize the computation of conjugate gradient method applied to normal equations for solving least squares problems. On modern architecture is always limited because of the global communication required for inner products. Inner products often therefore present a bottleneck, and it is desirable to reduce or even eliminate ...

Classification is one of the most important task in application areas of artificial neural networks (ANN).Training neural networks is a complex task in the supervised learning field of research. The main difficulty in adopting ANN is to find the most appropriate combination of learning, transfer and training function for the classification task. We compared the performances of three types of tr...

We consider gradient algorithms for minimizing a quadratic function in R with large n. We suggest a particular sequence of step-lengthes and demonstrate that the resulting gradient algorithm has a convergence rate comparable with that of Conjugate Gradients and other methods based on the use of Krylov spaces. When the problem is large and sparse, the proposed algorithm can be more efficient tha...

Conjugate gradient algorithms are very powerful methods for solving large-scale unconstrained optimization problems characterized by low memory requirements and strong local and global convergence properties. Over 25 variants of different conjugate gradient methods are known. In this paper we propose a fundamentally different method, in which the well known parameter k β is computed by an appro...

In this paper we suggest another accelerated conjugate gradient algorithm that for all both the descent and the conjugacy conditions are guaranteed. The search direction is selected as where , The coefficients 0 k ≥ 1 1 1 1 ( / ) ( / ) T T T T k k k k k k k k k k k k k k d g y g y s s t s g y s θ + + + + = − + − , s 1 1 ( ) k k g f x + + = ∇ 1 . k k k s x x + = − k θ and in this linear combinat...

In this paper we explore different strategies to guide backpropagation algorithm used for training artificial neural networks. Two different variants of steepest descent-based backpropagation algorithm, and four different variants of conjugate gradient algorithm are tested. The variants differ whether or not the time component is used, and whether or not additional gradient information is utili...

In this paper we suggest another accelerated conjugate gradient algorithm that for all both the descent and the conjugacy conditions are guaranteed. The search direction is selected as where , The coefficients 0 k ≥ 1 1 1 1 ( / ) ( / ) T T T T k k k k k k k k k k k k k k d g y g y s s t s g y s θ + + + + = − + − , s 1 1 ( ) k k g f x + + = ∇ 1 . k k k s x x + = − k θ and in this linear combinat...

On the Convergence Rate of Variants Conjugate Gradient Algorithm in Finite Precision Arithmetic

In this paper‎, ‎two extended three-term conjugate gradient methods based on the Liu-Storey ({tt LS})‎ ‎conjugate gradient method are presented to solve unconstrained optimization problems‎. ‎A remarkable property of the proposed methods is that the search direction always satisfies‎ ‎the sufficient descent condition independent of line search method‎, ‎based on eigenvalue analysis‎. ‎The globa...