We show that an LK proof of size m of a monotone sequent (a sequent that contains only formulas in the basis ∧,∨) can be turned into a proof containing only monotone formulas of size mO(log m) and with the number of proof lines polynomial in m. Also we show that some interesting special cases, namely the functional and the onto versions of PHP and a version of the Matching Principle, have polyn...

Optimization in Deep Learning is mainly guided by vague intuitions and strong assumptions, with a limited understanding how why these work practice. To shed more light on this, our provides some deeper understandings of SGD behaves empirically analyzing the trajectory taken from line search perspective. Specifically, costly quantitative analysis full-batch loss along trajectories common used mo...

Related DatabasesWeb of Science You must be logged in with an active subscription to view this.Article DataHistorySubmitted: 13 May 2020Accepted: 02 March 2021Published online: 20 2021Keywordslow-rank matrices, optimization on manifolds, multilevel optimization, Riemannian retraction-based line search, roundoff errorAMS Subject Headings65F10, 65N22, 65F50, 65K10Publication DataISSN (print): 106...

In this paper‎, ‎two extended three-term conjugate gradient methods based on the Liu-Storey ({tt LS})‎ ‎conjugate gradient method are presented to solve unconstrained optimization problems‎. ‎A remarkable property of the proposed methods is that the search direction always satisfies‎ ‎the sufficient descent condition independent of line search method‎, ‎based on eigenvalue analysis‎. ‎The globa...

In this paper we present a subgradient method with non-monotone line search for the minimization of convex functions simple constraints. Different from standard prefixed step sizes, new selects sizes in an adaptive way. Under mild conditions asymptotic convergence results and iteration-complexity bounds are obtained. Preliminary numerical illustrate relative efficiency proposed method.