The deep Boltzmann machine is a powerful model that extracts the hierarchical structure of observed data. While inference is typically slow due to its undirected nature, we argue that the emerging feature hierarchy is still explicit enough to be traversed in a feedforward fashion. The claim is corroborated by training a set of deep neural networks on real data and measuring the evolution of the...

Margin-Based Principle has been proposed for a long time, it has been proved that this principle could reduce the structural risk and improve the performance in both theoretical and practical aspects. Meanwhile, feed-forward neural network is a traditional classifier, which is very hot at present with a deeper architecture. However, the training algorithm of feed-forward neural network is devel...

Inhibitory interneurons across diverse brain regions commonly exhibit spontaneous spiking activity, even in the absence of external stimuli. It is not well understood how stimulus-evoked inhibition can be distinguished from background inhibition arising from spontaneous firing. We found that noradrenaline simultaneously reduced spontaneous inhibitory inputs and enhanced evoked inhibitory curren...

Given the presence of massive feedback loops in brain networks, it is difficult to disentangle the contribution of feed-forward and feedback processing to the recognition of visual stimuli, in this case, of emotional body expressions. The aim of the present work is to shed light on how well feed-forward processing explains rapid categorization of this important class of stimuli. By means of par...

This paper introduces a new method which employs the concept of “Orientation Vectors” to train a feed forward neural network. It is shown that this method is suitable for problems where large dimensions are involved and the clusters are characteristically sparse. For such cases, the new method is not NP hard as the problem size increases. We ‘derive’ the present technique by starting from Kolmo...

Perceptrons. A perceptron is a linear classifier of the form y = sign(σ i=1wixi+ b) where the weights w = (w1, . . . , wd) are trained using stochastic gradient descent. A perceptron is guaranteed to converge to some hyperplane separating two classes if the two classes are linearly separable (i.e., if there exists at least one hyperplane such that all points from Class 1 are on one side of it a...

There have been several attempts to mathematically understand neural networks and many more from biological and computational perspectives. The field has exploded in the last decade, yet neural networks are still treated much like a black box. In this work we describe a structure that is inherent to a feed forward neural network. This will provide a framework for future work on neural networks ...

Coordinated patterns of precisely timed action potentials (spikes) emerge in a variety of neural circuits but their dynamical origin is still not well understood. One hypothesis states that synchronous activity propagating through feed-forward chains of groups of neurons (synfire chains) may dynamically generate such spike patterns. Additionally, synfire chains offer the possibility to enable r...

We study semi-infinite and bi-infinite scalar feed-forward networks. We find that the temporal dynamics of these systems is closely linked to the spatial dynamics of an associated interval map and show how this interval map may be used to describe stationary interfaces. Beyond stationary structures, we show that the onset of instabilities in finite networks is intimately related to the emergenc...

In [18] the authors developed a method for computing normal forms of dynamical systems with a coupled cell network structure. We now apply this theory to one-parameter families of homogeneous feed-forward chains with 2-dimensional cells. Our main result is that Hopf bifurcations in such families generically generate branches of periodic solutions with amplitudes growing like ∼ |λ| 1 2 ,∼ |λ| 1 ...