Conjugate gradient methods have many advantages in real numerical experiments, such as fast convergence and low memory requirements. This paper considers a class of conjugate gradient learning methods for backpropagation (BP) neural networks with three layers. We propose a new learning algorithm for almost cyclic BP neural networks based on PRP conjugate gradient method. We then establish the d...

The techniques and analysis presented in this thesis provide new methods to solve optimization problems posed on Riemannian manifolds. These methods are applied to the subspace tracking problem found in adaptive signal processing and adaptive control. A new point of view is offered for the constrained optimization problem. Some classical optimization techniques on Euclidean space are generalize...

The conjugate gradient algorithm (CG) is an eeective tool for solving a system of linear equation with a positive deenite coeecient matrix. We show the reasons for a possible breakdown of the method when applied to a symmetric system with an indeenite coeecient matrix. Although in nite arithmetic a breakdown of the method occurs rather seldom, near breakdowns may slow down the speed of converge...

A new algorithm, the dual active set algorithm, is presented for solving a minimization problem with equality constraints and bounds on the variables. The algorithm identifies the active bound constraints by maximizing an unconstrained dual function in a finite number of iterations. Convergence of the method is established, and it is applied to convex quadratic programming. In its implementable...

The Polyhedral Active Set Algorithm (PASA) is designed to optimize a general nonlinear function over polyhedron. Phase one of the algorithm nonmonotone gradient projection algorithm, while phase two an active set that explores faces constraint A gradient-based implementation presented, where projected version conjugate employed in two. Asymptotically, only performed. Comparisons are given with ...

Computation of the inner state parameters in DSO inversion requires solving a large normal matrix system. A combined conjugate gradient and Lanczos iterative technique can be used to both solve the system and approximate some of the spectrum of the normal operator. At each iteration of the conjugate gradient algorithm, a small tridiagonal matrix (of dimension equal to the number of iterations) ...

Computation of the inner state parameters in DSO inversion requires solving a large normal matrix system. A combined conjugate gradient and Lanczos iterative technique can be used to both solve the system and approximate some of the spectrum of the normal operator. At each iteration of the conjugate gradient algorithm, a small tridiagonal matrix (of dimension equal to the number of iterations) ...

Variational Bayesian (VB) methods are typically only applied to models in the conjugate-exponential family using the variational Bayesian expectation maximisation (VB EM) algorithm or one of its variants. In this paper we present an efficient algorithm for applying VB to more general models. The method is based on specifying the functional form of the approximation, such as multivariate Gaussia...

A modified spectral PRP conjugate gradient method is presented for solving unconstrained optimization problems. The constructed search direction is proved to be a sufficiently descent direction of the objective function. With an Armijo-type line search to determinate the step length, a new spectral PRP conjugate algorithm is developed. Under some mild conditions, the theory of global convergenc...