Lyapunov functions for convergence of principal component algorithms

Recent theoretical analyses of a class of unsupervized Hebbian principal component algorithms have identified its local stability conditions. The only locally stable solution for the subspace P extracted by the network is the principal component subspace P∗. In this paper we use the Lyapunov function approach to discover the global stability characteristics of this class of algorithms. The subspace projection error, least mean squared projection error, and mutual information I are all Lyapunov functions for convergence to the principal subspace, although the various domains of convergence indicated by these Lyapunov functions leave some of P-space uncovered. A modification to I yields a ‘principal subspace information’ Lyapunov function I ′ with a domain of convergence which covers almost all of P-space. This shows that this class of algorithms converges to the principal subspace from almost everywhere.