Neural networks are usually trained by some form of stochastic gradient descent (SGD)). A number of strategies are in common use intended to improve SGD optimization, such as learning rate schedules, momentum, and batching. These are motivated by ideas about the occurrence of local minima at different scales, valleys, and other phenomena in the objective function. Empirical results presented he...

We analyze various scenarios of adaptive wave-front phase-aberration correction in optical-receiver-type systems when inhomogeneties of the wave propagation medium are either distributed along the propagation path or localized in a few thin layers remotely located from the receiver telescope pupil. Phase-aberration compensation is performed with closed-loop control architectures based on decoup...

The Burer-Monteiro [1] decomposition (X = Y Y T ) with stochastic gradient descent is commonly employed to speed up and scale up matrix problems including matrix completion, subspace tracking, and SDP relaxation. Although it is widely used in practice, there exist no known global convergence results for this method. In this paper, we prove that, under broad sampling conditions, a first-order ra...