A two-phase descent direction method for unconstrained stochastic optimization problem is proposed. A line search method with an arbitrary descent direction is used to determine the step sizes during the initial phase, and the second phase performs the stochastic approximation (SA) step sizes. The almost sure convergence of the proposed method is established, under standard assumption for desce...

<span style="color: #333333; font-family: Calibri, sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-dec...

In this report we present computational methods for the best multilinear rank approximation problem. We consider algorithms build on quasi-Newton methods operating on product of Grassmann manifolds. Specifically we test and compare methods based on BFGS and L-BFGS updates in local and global coordinates with the Newton-Grassmann and alternating least squares methods. The performance of the quas...

The alternating direction method of multipliers (ADMM) is an effective for solving convex problems from a wide range fields. At each iteration, the classical ADMM solves two subproblems exactly. However, in many applications, it expensive or impossible to obtain exact solutions subproblems. To overcome difficulty, some proximal terms are added This class methods typically original subproblem ap...

An algorithm for solving large nonlinear optimization problems with simple bounds is de scribed It is based on the gradient projection method and uses a limited memory BFGS matrix to approximate the Hessian of the objective function It is shown how to take advan tage of the form of the limited memory approximation to implement the algorithm e ciently The results of numerical tests on a set of l...

For many applications in graphics, one is confronted with the task of solving an optimization problem or solving a nonlinear system of equations. Efficient methods for solving these problems require derivatives to be available. Differentiation in numerical methods also occurs for many other reasons, such as computing forces from potential energy or computing normal and curvature information fro...

In previous work, the authors provided a foundation for the theory of variable metric proximal point algorithms in Hilbert space. In that work conditions are developed for global, linear, and super–linear convergence. This paper focuses attention on two matrix secant updating strategies for the finite dimensional case. These are the Broyden and BFGS updates. The BFGS update is considered for ap...

In this article, we explore the effectiveness of different numerical techniques in the training of backpropaqgation neural networks (BPNN) which are fed with wavelet-transformed data to capture useful information on various time scales. The purpose is to predict S&P500 future prices using BPNN trained with conjugate gradient (Fletcher-Reeves update, Polak-Ribiére update, Powell-Beale restart), ...

Problem statement: The Maximum Likelihood Estimation (MLE) technique is the most efficient statistical approach to estimate parameters in a cross-sectional model. Often, MLE gives rise to a set of non-linear systems of equations that need to be solved iteratively using the Newton-Raphson technique. However, in some situations such as in the Negative-Lindley distribution where it involves more t...

We propose a novel method for fitting planar B-spline curves to unorganized data points. In traditional methods, optimization of control points and foot points are performed in two very time-consuming steps in each iteration: 1) control points are updated by setting up and solving a linear system of equations; and 2) foot points are computed by projecting each data point onto a B-spline curve. ...