The focus of this study is how we can efficiently implement the neural network backpropagation algorithm on a network of computers (NOC) for concurrent execution. We assume a distributed system with heterogeneous computers and that the neural network is replicated on each computer. We propose an architecture model with efficient pattern allocation that takes into account the speed of processors...

Abstract This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning. The ReLU network can be analyzed by the ridgelet transform with respect to Lizorkin distributions. By showing three reconstruction formulas by using the Fourier slice the...

Many researchers consider a neural network to be a "black box" that maps the unknown relationships of inputs to corresponding outputs. By viewing neural networks in this manner, researchers often include many more input variables than are necessary for finding good solutions. This causes unneeded computation as well as impeding the search process by increasing the complexity of the network. The...

The rainfall-runoff relationship is one of the most complex hydrological phenomena. In recent years, hydrologists have successfully applied backpropagation neural network as a tool to model various nonlinear hydrological processes because of its ability to generalize patterns in imprecise or noisy and ambiguous input and output data sets. However, the backpropagation neural network convergence ...