Recently, important contributions on convergence studies of conjugate gradient methods have been made by Gilbert and Nocedal [6]. They introduce a “sufficient descent condition” to establish global convergence results, whereas this condition is not needed in the convergence analyses of Newton and quasi-Newton methods, [6] hints that the sufficient descent condition, which was enforced by their ...

The use of artificial neural networks has increased in many areas of engineering. In particular, this method has been applied to many geotechnical engineering problems and demonstrated some degree of success. A review of the literature reveals that it has been used successfully in modeling soil behavior, site characterization, earth retaining structures, settlement of structures, slope stabilit...

2 (First order) Descent methods, rates 2 2.1 Gradient descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.2 What can we achieve? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Second order methods: Newton’s method . . . . . . . . . . . . . . . . . 7 2.4 Multistep first order methods . . . . . . . . . . . . . . . . . . . . . . . . 8 2.4.1 Heavy ball method . ...

2 Method 1 2.1 Optical Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2.2 Lucas Kanade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.3 Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.4 Conjugate Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.5 Newton’s Method . . . . . . ...

Two new nonlinear spectral conjugate gradient methods for solving unconstrained optimization problems are proposed. One is based on the Hestenes and Stiefel (HS) method and the spectral conjugate gradient method. The other is based on a mixed spectral HS-CD conjugate gradient method, which combines the advantages of the spectral conjugate gradient method, the HS method, and the CD method. The d...

Classification is one of the most important task in application areas of artificial neural networks (ANN).Training neural networks is a complex task in the supervised learning field of research. The main difficulty in adopting ANN is to find the most appropriate combination of learning, transfer and training function for the classification task. We compared the performances of three types of tr...

In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. However, for some ill-conditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. A limited memory version of the nonlinear conjugate gradient method is developed. The memory is used to both detect the loss of o...