In this paper we propose a framework for developing globally convergent batch training algorithms with adaptive learning rate. The proposed framework provides conditions under which global convergence is guaranteed for adaptive learning rate training algorithms. To this end, the learning rate is appropriately tuned along the given descent direction. Providing conditions regarding the search dir...

This paper introduces a globally convergent block (column– and row–) cyclic Jacobi method for diagonalization of Hermitian matrices and for computation of the singular value decomposition of general matrices. It is shown that a block rotation (generalization of the Jacobi’s 2× 2 rotation) must be computed and implemented in a particular way to guarantee global convergence. This solves a long st...

We present new algorithms for penalized-likelihood image reconstruction: modified BSREM (block sequential regularized expectation maximization) and relaxed OS-SPS (ordered subsets separable paraboloidal surrogates). Both of them are globally convergent to the unique solution, easily incorporate convex penalty functions, and are parallelizable—updating all voxels (or pixels) simultaneously. They...

Positive results are proved here about the ability of numerical simulations to reproduce the exponentialmean-square stability of stochastic differential equations (SDEs). The first set of results applies under finite-time convergence conditions on the numerical method. Under these conditions, the exponential mean-square stability of the SDE and that of the method (for sufficiently small step si...

For many years globally convergent probability-one homotopy methods have been remarkably successful on difficult realistic engineering optimization problems, most of which were attacked by homotopy methods because other optimization algorithms failed or were ineffective. Convergence theory has been derived for a few particular problems, and considerable fixed point theory exists, but generally ...

In this paper, we present a nonmonotone trust-region algorithm for unconstrained optimization. We first introduce a variant of the nonmonotone strategy proposed by Ahookhosh and Amini cite{AhA 01} and incorporate it into the trust-region framework to construct a more efficient approach. Our new nonmonotone strategy combines the current function value with the maximum function values in some pri...

The mixed complementarity problem can be reformulated as a nonsmooth equation by using the median operator. In this paper, we rst study some useful properties of this reformulation and then derive the Chen-Harker-Kanzow-Smale smoothing function for the mixed complementarity problem. On the basis of this smoothing function, we present a smoothing Newton method for solving the mixed complementari...

Superlinear convergence of the Newton method for nonsmooth equations requires a “semismoothness” assumption. In this work we prove that locally Lipschitz functions definable in an o-minimal structure (in particular semialgebraic or globally subanalytic functions) are semismooth. Semialgebraic, or more generally, globally subanalytic mappings present the special interest of being γ-order semismo...

In this paper, a new globally convergent modification of the Resilient Propagation-Rprop algorithm is presented. This new addition to the Rprop family of methods builds on a mathematical framework for the convergence analysis that ensures that the adaptive local learning rates of the Rprop’s schedule generate a descent search direction at each iteration. Simulation results in six problems of th...