Large batch size training of Neural Networks has been shown to incur accuracy loss when trained with the current methods. The precise underlying reasons for this are still not completely understood. Here, we study large batch size training through the lens of the Hessian operator and robust optimization. In particular, we perform a Hessian based study to analyze how the landscape of the loss fu...

This short note considers Hessian updates for a sequential quadratically constrained quadratic programming (SQCQP) method. The SQCQP method is an iterative method to solve an inequality constrained optimization problem. At each iteration, the SQCQP method solves a quadratically constrained quadratic programming subproblem whose objective function and constraints are approximations to the object...

In this work we develop Curvature Propagation (CP), a general technique for efficiently computing unbiased approximations of the Hessian of any function that is computed using a computational graph. At the cost of roughly two gradient evaluations, CP can give a rank-1 approximation of the whole Hessian, and can be repeatedly applied to give increasingly precise unbiased estimates of any or all ...

Resistance in wheat (Triticum aestivum) to the Hessian fly (Mayetiola destructor), a major insect pest of wheat, is based on a gene-for-gene interaction. Close linkage (3 +/- 2 cM) was discovered between Hessian fly avirulence genes vH3 and vH5. Bulked segregant analysis revealed two DNA markers (28-178 and 23-201) within 10 cM of these loci and only 3 +/- 2 cM apart. However, 28-178 was locate...

In this paper we propose a novel approach to measuring curvature in color or vector-valued images (up to 4-dimensions) based on quaternion singular value decomposition of a Hessian matrix. This approach generalizes the existing scalar-image curvature approach which makes use of the eigenvalues of the Hessian matrix [1]. In the case of vector-valued images, the Hessian is no longer a 2D matrix b...

The Hessian fly, Mayetiola destructor, is a serious pest of wheat and an experimental organism for the study of gall midge-plant interactions. In addition to food digestion and detoxification, the gut of Hessian fly larvae is also an important interface for insect-host interactions. Analysis of the genes expressed in the Hessian fly larval gut will enhance our understanding of the overall gut p...

We give explicit formulas for the number of distinct elliptic curves over a finite field, up to isomorphism, in the families of Legendre, Jacobi, Hessian and generalized Hessian curves.

We address the inverse problem for scattering of acoustic waves due to an inhomogeneous medium. We derive and analyze the Hessian in both Hölder and Sobolev spaces. Using an integral equation approach based on Newton potential theory and compact embeddings in Hölder and Sobolev spaces, we show that the Hessian can be decomposed into two components, both of which are shown to be compact operator...

Let z = (z1, · · · , zn) and ∆ = ∑n i=1 ∂ 2 ∂z i the Laplace operator. The main goal of the paper is to show that the wellknown Jacobian conjecture without any additional conditions is equivalent to the following what we call vanishing conjecture: for any homogeneous polynomial P (z) of degree d = 4, if ∆P(z) = 0 for all m ≥ 1, then ∆P(z) = 0 when m >> 0, or equivalently, ∆P(z) = 0 when m > 3 2...