Stability and Convergence of Stochastic Approximation using the O.D.E. Method - Decision and Control, 1998. Proceedings of the 37th IEEE Conference on

It is shown here that stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated 0.d.e. This in turn implies convergence of the algorithm. Several specific classes of algorithms are considered as applications. It is found that the results provide (i) a simpler derivation of known results for reinforcement learning algorithms; (ii) a proof for the first time that a class of asynchronous stochastic approximation algorithms are convergent without using any a priori assumption of stability. (iii) a proof for the first time that asynchronous adaptive critic and &-learning algorithms are convergent for the average cost optimal control problem.