It is well known that when a connectionist network is trained on one set of patterns and then attempts to add new patterns to its repertoire, catastrophic interference may result. The use of sparse, orthogonal hidden-layer representations has been shown to reduce catastrophic interference. The author demonstrates that the use of sparse representations may, in certain cases, actually result in w...

Learning invariant representations is an important problem in machine learning and pattern recognition. In this paper, we present a novel framework of transformationinvariant feature learning by incorporating linear transformations into the feature learning algorithms. For example, we present the transformation-invariant restricted Boltzmann machine that compactly represents data by its weights...

Recent years have seen an increasing interest in sparseness constraints for image classification and object recognition, probably motivated by the evidence of sparse representations internal in the primate visual cortex. It is still unclear, however, whether or not sparsity helps classification. In this paper we analyze the image classification task on CIFAR-10, a very challenging dataset, and ...