Hans Peter Graf AT&T Bell Laboratories Holmdel, NJ 07733 The back propagation algorithm has been modified to work without any multiplications and to tolerate comput.ations with a low resolution, which makes it. more attractive for a hardware implementatioll. Numbers are represented in float.ing point format with 1 bit mantissa and 3 bits in the exponent for the states, and 1 bit mantissa and 5 ...

This report provides detailed description and necessary derivations for the BackPropagation Through Time (BPTT) algorithm. BPTT is often used to learn recurrent neural networks (RNN). Contrary to feed-forward neural networks, the RNN is characterized by the ability of encoding longer past information, thus very suitable for sequential models. The BPTT extends the ordinary BP algorithm to suit t...