This paper is devoted to obtaining an averaging principle for systems of slow-fast stochastic differential equations, where the fast variable drift is periodically modulated on a fast time-scale. The approach developed here combines probabilistic methods with a recent analytical result on long-time behavior for second order elliptic equations with time-periodic coefficients.

We study balanced model reduction for stable bilinear systems in the limit of partly vanishing Hankel singular values. We show that the dynamics can be split into a fast and a slow subspace and prove an averaging principle for the slow dynamics. We illustrate our method with an example from stochastic control (density evolution of a dragged Brownian particle) and discuss issues of structure pre...

In the presence of multiscale dynamics in a reaction network, direct simulation methods become inefficient as they can only advance the system on the smallest scale. This work presents stochastic averaging techniques to accelerate computations for obtaining estimates of expected values and sensitivities with respect to the steady state distribution. A two-time-scale formulation is used to estab...

Radiation transport in a one dimensional random medium is considered. The Karhunen-Loève spectral expansion method is shown to provide an efficient representation of the total cross section that is a continuous random process in space. Numerical results for an exponential covariance function show that a low order truncation suffices to capture the dominant components of the random process for r...

We consider the averaging principle for stochastic reaction–diffusion equations. Under some assumptions providing existence of a unique invariant measure of the fast motion with the frozen slow component, we calculate limiting slow motion. The study of solvability of Kolmogorov equations in Hilbert spaces and the analysis of regularity properties of solutions, allow to generalize the classical ...

Consider a stochastic differential equation whose diffusion vector fields are formed from an integrable family of Hamiltonian functions Hi, i = 1, . . . n. We investigate the effect of a small transversal perturbation of order to such a system. An averaging principle is shown to hold for this system and the action component of the solution converges, as → 0, to the solution of a deterministic s...

We consider regularized stochastic learning and online optimization problems, where the objective function is the sum of two convex terms: one is the loss function of the learning task, and the other is a simple regularization term such as l1-norm for promoting sparsity. We develop a new online algorithm, the regularized dual averaging (RDA) method, that can explicitly exploit the regularizatio...

Time series averaging in dynamic time warping (DTW) spaces has been successfully applied to improve pattern recognition systems. This article proposes and analyzes subgradient methods for the problem of finding a sample mean in DTW spaces. The class of subgradient methods generalizes existing sample mean algorithms such as DTW Barycenter Averaging (DBA). We show that DBA is a majorize-minimize ...

Using a simple stochastic growth model, this paper demonstrates that the coefficient of variation of aggregate output or GDP does not necessarily go to zero even if the number of sectors or economic agents goes to infinity. This phenomenon known as non-self-averaging implies that even if the number of economic agents is large, dispersion can remain significant, and, therefore, that we can not l...

Stochastic Gradient Descent (SGD) is one of the simplest and most popular stochastic optimization methods. While it has already been theoretically studied for decades, the classical analysis usually required nontrivial smoothness assumptions, which do not apply to many modern applications of SGD with non-smooth objective functions such as support vector machines. In this paper, we investigate t...