A stochastic algorithm for the recursive approximation of the location θ of a maximum of a regression function has been introduced by Kiefer and Wolfowitz (1952) in the univariate framework, and by Blum (1954) in the multivariate case. The aim of this paper is to provide a companion algorithm to the Kiefer-Wolfowitz-Blum algorithm, which allows to simultaneously recursively approximate the size...

We present a new class of perceptron-like algorithms with margin in which the “effective” learning rate, defined as the ratio of the learning rate to the length of the weight vector, remains constant. We prove that the new algorithms converge in a finite number of steps and show that there exists a limit of the parameters involved in which convergence leads to classification with maximum margin.