| In classic backpropagation nets, as introduced by Rumelhart et al. 1], the weights are modiied according to the method of steepest descent. The goal of this weight modiication is to minimise the error in net-outputs for a given training set. Basing upon Jacobs' work 2], we point out drawbacks of steepest descent and suggest improvements on it. These yield a backpropagation net, which adjusts ...

In this paper we use steepest descent method for solving zero-one nonlinear programming problem. Using penalty function we transform this problem to an unconstrained optimization problem and then by steepest descent method we obtain the original problem optimal solution. 2007 Elsevier Inc. All rights reserved.

We consider the special case of the restarted Arnoldi method for approximating the product of a function of a Hermitian matrix with a vector which results when the restart length is set to one. When applied to the solution of a linear system of equations, this approach coincides with the method of steepest descent. We show that the method is equivalent with an interpolation process in which the...

Classical gradient methods and evolutionary algorithms represent two very different classes of optimization techniques that seem to have very different properties. This paper discusses some aspects of some “obvious” differences and explores to what extent a hybrid method, the evolutionary-gradient-search procedure, can be used beneficially in the field of continuous parameter optimization. Simu...

A recently published idea is to use the A*-Algorithm to optimize the topology of Neural Networks. In this paper, optimization techniques are investigated that combine the A*-Algorithm with diierent parallel training algorithms, namely the backpropagation algorithm and several hybrid algorithms. The hybrid algorithms combine the backpropagation's steepest descent method with diierent sets of gen...

The application of a reasonable selection scheme plays an essential role in virtually all evolutionary algorithms. By not considering less-fit individuals, however, the algorithm discards valuable information about the fitness function. This paper explores to which extent a hybrid method, the evolutionary-gradient-search procedure, that applies a global operator to all offspring can be benefici...

The possibilities inherent in steepest descent methods have been considerably amplified by the introduction of the Barzilai-Borwein choice of step-size, and other related ideas. These methods have proved to be competitive with conjugate gradient methods for the minimization of large dimension unconstrained minimization problems. This paper suggests a method which is able to take advantage of th...

We consider the special case of the restarted Arnoldi method for approximating the product of a function of a Hermitian matrix with a vector which results when the restart length is set to one. When applied to the solution of a linear system of equations, this approach coincides with the method of steepest descent. We show that the method is equivalent with an interpolation process in which the...