4 Estimates 9 4.1 The zeroth order and volume estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.2 A partial second order estimate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.3 Gradient estimates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.4 The second order estimates . . . . ....

Saddle-point problems have recently gained increased attention from the machine learning community, mainly due to applications in training Generative Adversarial Networks using stochastic gradients. At same time, some only a zeroth-order oracle is available. In this paper, we propose several algorithms solve smooth (strongly) convex-concave saddle-point oracles and estimate their convergence ra...

We consider the problem of global optimization of an unknown non-convex smooth function with zeroth-order feedback. In this setup, an algorithm is allowed to adaptively query the underlying function at different locations and receives noisy evaluations of function values at the queried points (i.e. the algorithm has access to zeroth-order information). Optimization performance is evaluated by t...

In this paper, we consider a stochastic distributed nonconvex optimization problem with the cost function being over n agents having access only to zeroth-order (ZO) information of cost. This has various machine learning applications. As solution, propose two ZO algorithms, in which at each iteration agent samples local oracle points time-varying smoothing parameter. We show that proposed algor...

We consider stochastic zeroth-order optimization over Riemannian submanifolds embedded in Euclidean space, where the task is to solve problems with only noisy objective function evaluations. Toward this, our main contribution propose estimators of gradient and Hessian from evaluations, based on a version Gaussian smoothing technique. The proposed overcome difficulty nonlinearity manifold constr...

We consider the problem of minimizing a smooth, Lipschitz, convex function over compact, set using sub-zeroth-order oracles: an oracle that outputs sign directional derivative for given point and direction, compares values pair points, noisy value point. show sample complexity optimization these oracles is polynomial in relevant parameters. The algorithm we provide comparator first with known r...