We analyze multilayer neural networks in the asymptotic regime of simultaneously (a) large network sizes and (b) numbers stochastic gradient descent training iterations. rigorously establish limiting behavior output. The limit procedure is valid for any number hidden layers, it naturally also describes loss. ideas that we explore are to take limits each layer sequentially characterize evolution...

This work provides new results for the analysis of random sequences in terms ? p -compressibility. The characterize degree which a sequence can be approximated by its best k -sparse version under different rates significant coefficients (compressibility analysis). In particular, notion strong -characterization is introduced to denote that has well-defined asymptotic limit (sample-wise) -term ap...