Consensus optimization has received considerable attention in recent years. A number of decentralized algorithms have been proposed for convex consensus optimization. However, on consensus optimization with nonconvex objective functions, our understanding to the behavior of these algorithms is limited. When we lose convexity, we cannot hope for obtaining globally optimal solutions (though we st...

2 Method 1 2.1 Optical Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2.2 Lucas Kanade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2.3 Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.4 Conjugate Gradient Descent . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.5 Newton’s Method . . . . . . ...

Using search directions of a recent class of three--term conjugate gradient methods, modified versions of the Hestenes-Stiefel and Polak-Ribiere-Polyak methods are proposed which satisfy the sufficient descent condition. The methods are shown to be globally convergent when the line search fulfills the (strong) Wolfe conditions. Numerical experiments are done on a set of CUTEr unconstrained opti...

The method of conjugate gradients provides a very effective way to optimize large, deterministic systems by gradient descent. In its standard form, however, it is not amenable to stochastic approximation of the gradient. Here we explore ideas from conjugate gradient in the stochastic (online) setting, using fast Hessian-gradient products to set up low-dimensional Krylov subspaces within individ...