A Catalog of Elemental Neural Circuits

It is well-known that abstract neurons of the McCulloch-Pitts type compute input-output functions that are formally equivalent to suitably defined Boolean functions. Boolean functions, in turn, can be classified (up to isomorphism of underlying logical structure) into a limited number of basic types, each of which is non-isomorphic to each other type. These basic types are conveniently expressed by means of a canonic logical formula involving a minimal number of constituent operations. Each such canonic formula can be translated into an equivalent representation as a neural circuit, resulting in a catalog of basic, distinct neural circuits. Each circuit in this catalog computes a distinct Boolean function, and every Boolean function is computed by one such network, making this a kind of “periodic table” of elemental neural circuits. This brief article presents and discusses this catalog of circuits. An intriguing possibility raised by this analysis is that neural circuits in biological networks might, like the circuits in this catalog, tend to take forms that are minimally complex given the functions they compute—a kind of “principle of neural parsimony.” The modern mathematical investigation of the properties of neural networks began with McCulloch and Pitts’s model of an “abstract neuron,” which sums D binary inputs, each either excitatory or inhibitory, and fires if and only if the total excitation meets or exceeds a predefined threshold θ. McCulloch and Pitts showed that networks of these simple devices were of extraordinary computational power, in fact equivalent to that of a Turing Machine or modern digital computer (McCulloch & Pitts, 1943). Underlying their proof is the idea that any abstract neuron, or circuit of abstract neurons, computes a Boolean function: that is, a mapping from D binary variables (the inputs on the abstract “dendrites”) to one variable (the output on the abstract “axon”). The mathematics of neural networks are thus inextricably tied up with the mathematics of Boolean functions. A similar relation exists between electrical circuit diagrams, of the type that illustrate textbooks on electrical engineering, and logical formulae, as one would find in a textbook of mathematical logic. The two representations are in a very I am grateful to Randy Gallistel for helpful discussions and to NSF SBR-9875175 for support. Please direct comments to the author at [email protected].