Low - Power Stochastic Arithmetic Feed - Forward Neural Network

I thank my advisor Yngvar Berg for accepting me as one of his students, and so making this work possible, and for introducing me to the very interesting field of neural computation. I will also thank him for being encouraging, and allowing me a great freedom in choice of methods and solutions. Many thanks to Tor Sverre Lande as well, for his constructive critisism of circuit implementations. Contents 1 Introduction 1 1.1 Biological neural networks-a brief 3 Stochastic computing network elements 13 3.1 Representing values with pulse 4 Output error generation 29 4.1 Integration and regeneration iii iv CONTENTS 5 Long term synaptic weight storage 39 5.1 Analog storage 6 Composition and performance of a complete network 57 6.1 Assembling the network parts into a 3-6-2 A Die photo of the chip 71 B Miscellaneous 73 B.1 Program for empirical calculation of the neuron transfer characteristic : 73 B.2 Op-amp integrator that implements a moving v vi LIST O F FIGURES 6.1 Floorplan of a 3-6-2 CHAPTER 1 Introduction In the traditional computational paradigm, which was introduced by von Neumann, problems are solved by an ordered set of instructions known as a program. These instructions are fed to the CPU in a sequential order, and processed in this order. An instruction typically just do a simple arithmetic operation or change the location of where to fetch the next instruction – a jump-instruction. To solve complex problems one usually have to process the same set of instructions thousands of times inside tight loops, that can be nested inside other tight loops. This means that it can take billions of instructions to solve such problems. Even the fastest supercomputers existing today, running advanced AI systems, can not perform real time visual or auditory recognition like you and me, but even simple mammals perform such visual and auditory recognition constantly. The biological brains ability to perform very complex tasks, with a minimum set of activities and resources, has inspired scientists to study the field of neural computation, and to try to implement artificial neural systems. There are also other properties associated with the biological brain that are desirable. It is very fault tolerant, and even though there are dying nerve cells every minute, it is still functioning without significant loss of performance. It is not like a digital computer where one damaged transistor can make the whole system go down. It is also …