Neural networks have recently re-emerged as a powerful hypothesis class, yielding impressive classification accuracy in multiple domains. However, their training is a non-convex optimization problem which poses theoretical and practical challenges. Here we address this difficulty by turning to “improper” learning of neural nets. In other words, we learn a classifier that is not a neural net but...

This paper describes the implementation of a fast neural net simulator on a novel parallel distributed-memory computer. A 60-processor system, named MUSIC, 1 is operational and runs the back-propagation algorithm at a speed of 247 million connection updates per second (continuous weight update) using 32 bit oating-point precision. This is equal to 1 GGops sustained performance. The complete sys...

Frequently, sequences of state transitions are triggered by specific signals. Learning these triggered sequences with recurrent neural networks implies storing them as different attractors of the recurrent hidden layer dynamics. A challenging test and also useful for application is conditional prediction of sequences giving just the trigger signal as an input and letting the recurrent neural ne...

This paper proposes a non-recurrent training algorithm, resilient propagation, for the Simultaneous Recurrent Neural network operating in relaxation-mode for computing high quality solutions of static optimization problems. Implementation details related to adaptation of the recurrent neural network weights through the non-recurrent training algorithm, resilient backpropagation, are formulated ...