Recurrent neural networks can be trained to be maximum a posteriori probability classifiers

This paper proves that supervised learning algorithms used to train recurrent neural networks have an equilibrium point when the network implements a Maximum A Posteriori Probability (MAP) classiier. The result holds as a limit when the size of the training set goes to innnity. The result is general, since it stems as a property of cost minimizing algorithms, but to prove it we implicitly assume that the network we are training has enough computing power to actually implement the MAP classiier. This assumption can be satissed using a universal dynamic system approximator. We refer our discussion to Block Feedback Neural Networks (B F Ns) and show that they actually have the universal approximation property.