Spatial patterns of neuronal activity arise in a variety of experimental studies. Previous theoretical work has demonstrated that a synaptic architecture featuring recurrent excitation and long-range inhibition can support sustained, spatially patterned solutions in integrodifferential equation models for activity in neuronal populations. However, this architecture is absent in some areas of th...

Convolutional Neural Network(CNN) has been widely used for image recognition with great success. However, there are a number of limitations of the current CNN based image recognition paradigm. First, the receptive field of CNN is generally fixed, which limits its recognition capacity when the input image is very large. Second, it lacks the computational scalability for dealing with images with ...

In this paper we show that programming languages are implementable on neural nets, namely, neural nets can be designed to solve any (computable) high level programming task. Constructions like the one that follows can also be used to built large scale neural nets that integrate learning and control structures. We use a very simple model of analog recurrent neural nets and a number-theoretic app...

Inspired by the principles of speed reading, we introduce Skim-RNN, a recurrent neural network (RNN) that dynamically decides to update only a small fraction of the hidden state for relatively unimportant input tokens. Skim-RNN gives computational advantage over an RNN that always updates the entire hidden state. Skim-RNN uses the same input and output interfaces as a standard RNN and can be ea...

Neocortical circuits are dominated by massive excitatory feedback: more than eighty percent of the synapses made by excitatory cortical neurons are onto other excitatory cortical neurons. Why is there such massive recurrent excitation in the neocortex and what is its role in cortical computation? Recent neurophysiological experiments have shown that the plasticity of recurrent neocortical synap...

Topographically organized neurons represent multiple stimuli within complex visual scenes and compete for subsequent processing in higher visual centers. The underlying neural mechanisms of this process have long been elusive. We investigate an experimentally constrained model of a midbrain structure: the optic tectum and the reciprocally connected nucleus isthmi. We show that a recurrent antit...

In this paper, we propose a novel neural architecture that adaptively learns an input-output mapping using both supervised and non-supervised trainings. This neural architecture consists of a combination of an ART2 (Adaptive Resonance Theory) neural network and recurrent neural networks. For this end, we developed an Extended Kalman Filter (EKF) based training algorithm for the involved recurre...

Simple recurrent networks were trained with sequences of phonemes from a corpus of Turkish words. The network's task was to predict the next phoneme. The aim of the study was to look at the representations developed within the hidden layer of the network in order to investigate the extent to which such networks can learn phonological regularities from such input. It was found that in the diiere...

There has been an increased interest in combining fuzzy systems with neural networks because fuzzy neural systems merge the advantages of both paradigms. On the one hand, parameters in fuzzy systems have clear physical meanings and rule-based and linguistic information can be incorporated into adaptive fuzzy systems in a systematic way. On the other hand, there exist powerful algorithms for tra...

Hybrid methods that utilize both content and rating information are commonly used in many recommender systems. However, most of them use either handcrafted features or the bag-of-words representation as a surrogate for the content information but they are neither effective nor natural enough. To address this problem, we develop a collaborative recurrent autoencoder (CRAE) which is a denoising r...