ویراساینس.
سند علمی

Finding Approximate Local Minima for Nonconvex Optimization in Linear Time

Naman Agarwal, Zeyuan Allen Zhu, Brian Bullins, Elad Hazan, Tengyu Ma

چکیده

We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which is linear in the input representation. The previously fastest methods run in time proportional to matrix inversion or worse. The time complexity of our algorithm to find a local minimum is even faster than that of gradient descent to find a critical point (which can be a saddle point). Our algorithm applies to a very general class of optimization problems including training a neural network and many other non-convex objectives arising in machine learning. ar X iv :1 61 1. 01 14 6v 1 [ m at h. O C ] 3 N ov 2 01 6

اسناد مشابه

  • Robust MIMO radar target localization via nonconvex optimization

    سند علمی۲۰۱۶Signal ProcessingJunli Liang, Dong Wang, Li Su, Badong Chen

    This paper addresses the problem of robust target localization in distributed multipleinput multiple-output (MIMO) radar using possibly outlier range measurements. To achieve robustness against outliers, we construct an objective function for MIMO target localization via the maxi

  • A Batch, Derivative-Free Algorithm for Finding Multiple Local Minima

    سند علمی۲۰۱۵Jeffrey Larson, Stefan M. Wild

    We propose a derivative-free algorithm for finding high-quality local minima for functions that require significant computational resources to evaluate. Our algorithm efficiently utilizes the computational resources allocated to it and also has strong theoretical results, almost

  • Communication Efficient, Sample Optimal, Linear Time Locally Private Discrete Distribution Estimation

    سند علمی۲۰۱۸CoRRJayadev Acharya, Ziteng Sun, Huanyu Zhang

    We consider discrete distribution estimation over k elements under ε-local differential privacy from n samples. The samples are distributed across users who send privatized versions of their sample to the server. All previously known sample optimal algorithms require linear (in k

  • Finding Approximate Solutions for Large Scale Linear Programs

    سند علمی۱۹۷۸Fabian A. Chudak

    Linear Programming is one of the most frequently applied tools for modeling and solving real world optimization problems. Nonetheless, most commercially available solvers are often incapable of dealing with large problem sizes, e.g. millions of variables or hundreds of thousands

  • Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization

    سند علمی۲۰۱۴NIPSMeisam Razaviyayn, Mingyi Hong, Zhi-Quan Luo, Jong-Shi Pang

    Consider the problem of minimizing the sum of a smooth (possibly non-convex) and a convex (possibly nonsmooth) function involving a large number of variables. A popular approach to solve this problem is the block coordinate descent (BCD) method whereby at each iteration only one

  • Design and Implementation of Compensator for Optimizing Linear Time Invariant System Parameters

    سند علمی۲۰۱۳Harikrishna Paik

    Most of control systems in vogue are Linear Time Invariant (LTI) systems and are characterized by several parameters such as settling time, over shoot and rise time etc. Optimization of these parameters is required to meet the desired system response and smooth operation of the s