We introduce a quasi-Newton method with block updates called Block BFGS. We show that this method, performed with inexact Armijo-Wolfe line searches, converges globally and superlinearly under the same convexity assumptions as BFGS. We also show that Block BFGS is globally convergent to a stationary point when applied to non-convex functions with bounded Hessian, and discuss other modifications...

Stochastic convex optimization is a basic and well studied primitive in machine learning. It is well known that convex and Lipschitz functions can be minimized efficiently using Stochastic Gradient Descent (SGD). The Normalized Gradient Descent (NGD) algorithm, is an adaptation of Gradient Descent, which updates according to the direction of the gradients, rather than the gradients themselves. ...

In this paper we address the question of how many objective functions are really needed to decide whether a given point is Pareto optimal. We prove a reduction result for the case of quasi-convex objective functions and a convex feasible set. This result states that in order to decide whether a point x in the decision space is Pareto optimal it suuces to consider at most n + 1 objectives at a t...

Throughout this paper, let Z, E, and F be topological vector spaces, let X ⊆ E and Y ⊆ F be nonempty, closed, and convex subsets. Let D : X → 2X , T : X → 2Y and Ψ : X ×Y × X → 2Z be set-valued mappings, and let C : X → 2Z be a set-valued mapping such that C(x) is a closed pointed and convex cone with intC(x) = ∅ for each x ∈ X , where intC(x) denotes the interior of the set C(x). Then the gene...