Traditional approaches to solving stochastic optimal control problems involve dynamic programming, and solving certain optimality equations. When recast as stochastic programming problems, structural aspects such as convexity are regained, and solution procedures based on decomposition and duality may be exploited. This paper explores a class of stationary, infinite-horizon stochastic optimizat...

We propose and analyze a variant of the classic Polyak-Ruppert averaging scheme, broadly used in stochastic gradient methods. Rather than a uniform average of the iterates, we consider a weighted average, with weights decaying in a geometric fashion. In the context of linear least squares regression, we show that this averaging scheme has a the same regularizing effect, and indeed is asymptotic...

The distributed averaging problem is to devise a protocol which will enable the members of a group of n > 1 agents to asymptotically determine in a decentralized manner, the average of the initial values of their scalar agreement variables. A typical averaging protocol can be modeled by a linear iterative equation whose update matrices are doubly stochastic. Building on the ideas proposed in [1...