Inspired by the recent paper (L. Ying, Journal of Scientific Computing, 84, 1–14 (2020), we explore relationship between mirror descent and variable metric method. When in decent is induced a convex function, whose Hessian close to objective this method enjoys both robustness from superlinear convergence for Newton type methods. applied linearly constrained minimization problem, prove global lo...

Several recent empirical studies demonstrate that important machine learning tasks such as training deep neural networks, exhibit a low-rank structure, where most of the variation in loss function occurs only few directions input space. In this paper, we leverage structure to reduce high computational cost canonical gradient-based methods gradient descent (GD). Our proposed Low-Rank Gradient De...

The steepest descent method has a rich history and is one of the simplest and best known methods for minimizing a function. While the method is not commonly used in practice due to its slow convergence rate, understanding the convergence properties of this method can lead to a better understanding of many of the more sophisticated optimization methods. Here, we give a short introduction and dis...