A Riemannian mean field formulation for two-layer neural networks with batch normalization

The training dynamics of two-layer neural networks with batch normalization (BN) is studied. It written as the a network without BN on Riemannian manifold. Therefore, we identify BN’s effect changing metric in parameter space. Later, infinite-width limit considered, and mean-field formulation derived for dynamics. shown to be Wasserstein gradient flow Theoretical analysis provided well-posedness convergence flow.