This paper presents the hybrid algorithm of global optimization of dynamic learning rate for multilayer feedforward neural networks (MLFNN). The effect of inexact line search on conjugacy was studied and a generalized conjugate gradient method based on this effect was proposed and shown to have global convergence for error backpagation of MLFNN. The descent property and global convergence was g...

The natural gradient method has been used effectively in conjugate Gaussian process models, but the non-conjugate case has been largely unexplored. We examine how natural gradients can be used in non-conjugate stochastic settings, together with hyperparameter learning. We conclude that the natural gradient can significantly improve performance in terms of wall-clock time. For illconditioned pos...

A modification of the Dai-Yuan conjugate gradient algorithm is proposed. Using the exact line search, the algorithm reduces to the original version of the Dai and Yuan computational scheme. For inexact line search the algorithm satisfies both the sufficient descent and conjugacy condition. A global convergence result is proved when the Wolfe line search conditions are used. Computational result...

In this paper the use of iterative methods for the solution of the carrier continuity equations in three-dimensional semiconductor device simulators is summarized. An overview of the derivation of the linear systems from the basic stationary semiconductor device equations is given and the algebraic properties of the nonsymmetric coefficient matrices are discussed. Results from the following cla...

The smoothing procedure has been introduced by Schh onauer as an acceleration algorithm for the Generalized Conjugate Gradient methods. In this paper, after giving our deenition of Conjugate Gradient type methods, and using a Generalized Hessenberg process, we introduce a Hybrid generalized Conjugate Gradient type method. We show that this Hybrid procedure is a variable smoothing procedure. And...

Conjugate gradient methods are efficient for smooth optimization problems, while there are rare conjugate gradient based methods for solving a possibly nondifferentiable convex minimization problem. In this paper by making full use of inherent properties of Moreau-Yosida regularization and descent property of modified conjugate gradient method we propose a modified Fletcher-Reeves-type method f...

in this paper, chebyshev acceleration technique is used to solve the fuzzy linear system (fls). this method is discussed in details and followed by summary of some other acceleration techniques. moreover, we show that in some situations that the methods such as jacobi, gauss-sidel, sor and conjugate gradient is divergent, our proposed method is applicable and the acquired results are illustrate...

Using search directions of a recent class of three--term conjugate gradient methods, modified versions of the Hestenes-Stiefel and Polak-Ribiere-Polyak methods are proposed which satisfy the sufficient descent condition. The methods are shown to be globally convergent when the line search fulfills the (strong) Wolfe conditions. Numerical experiments are done on a set of CUTEr unconstrained opti...

In this paper, Chebyshev acceleration technique is used to solve the fuzzy linear system (FLS). This method is discussed in details and followed by summary of some other acceleration techniques. Moreover, we show that in some situations that the methods such as Jacobi, Gauss-Sidel, SOR and conjugate gradient is divergent, our proposed method is applicable and the acquired results are illustrate...

In this preliminary work, left and right conjugate vectors are deened for nonsymmetric, nonsin-gular matrices A and some properties of these vectors are studied. A left conjugate direction (LCD) method for solving nonsymmetric systems of linear equations is proposed. The method reduces to the usual conjugate gradient method when A is symmetric positive deenite. A nite termination property of th...