In many machine learning problems such as the dual form of SVM, the objective function to be minimized is convex but not strongly convex. This fact causes difficulties in obtaining the complexity of some commonly used optimization algorithms. In this paper, we proved the global linear convergence on a wide range of algorithms when they are applied to some non-strongly convex problems. In partic...

We develop a general convergence theory for the generalized minimal residual method for least squares problems preconditioned with inner iterations. The inner iterations are performed by stationary iterative methods. We also present theoretical justifications for using specific inner iterations such as the Jacobi and SOR-type methods. The theory is improved particularly in the rankdeficient cas...

‎In this paper a new family of fifteenth-order methods with high efficiency index is presented‎. This family include four evaluations of the function and one evaluation of its first derivative per iteration.‎ ‎Therefore‎, ‎this family of methods has the efficiency index which equals 1.71877‎. ‎In order to show the applicability and validity of the class‎, ‎some numerical examples are discussed‎.

In this paper, we present two new families of third-order and fourth-order methods for finding multiple roots of nonlinear equations. Each of them requires one evaluation of the function and two of its first derivative per iteration. Several numerical examples are given to illustrate the performance of the presented methods.    

Projection type methods are among the most important methods for solving monotone linear variational inequalities. In this note, we analyze the iteration complexity for two projection methods and accordingly establish their worst-case O(1/t) convergence rates measured by the iteration complexity in both the ergodic and nonergodic senses, where t is the iteration counter. Our analysis does not r...

Based on the previous research results on the diffusion approach [1] to solve fixed point problem in linear algebra, we propose here a new analysis of the convergence speed of different iteration methods. In Section II, we define the iteration methods that are considered. Section III shows how to define the associated equivalent diffusion iteration. Section IV shows few examples to illustrate t...

A fully parallel version of the Contact dynamics (CD) method is presented in this paper. For large enough systems, 100% efficiency is reached for an arbitrary number of processors using hierarchical domain decomposition with dynamical load balancing. The iterative scheme to calculate the contact forces is left domain-wise sequential, with data exchange after each iteration step, which ensures i...

More than a decade ago, Bramble, Pasciak and Xu developed a framework in analyzing the multigrid methods with nonnested spaces or noninherited quadratic forms. It was subsequently known as the BPX multigrid framework, which was widely used in the analysis of multigrid and domain decomposition methods. However, the framework has an apparent limit in the analysis of nonnested V-cycle methods, and...

More than a decade ago, Bramble, Pasciak and Xu developed a framework in analyzing the multigrid methods with nonnested spaces or noninherited quadratic forms. It was subsequently known as the BPX multigrid framework, which was widely used in the analysis of multigrid and domain decomposition methods. However, the framework has an apparent limit in the analysis of nonnested V-cycle methods, and...

Tail-biting trellises are defined on a circular time axis and can have a smaller number of states than equivalent conventional trellises. Existing circular Viterbi algorithms (CVAs) on the tail-biting trellis are non-convergent and sub-optimal. In this study, we show that the net path metric of each tail-biting path is lower-bounded during the decoding process of the CVA. This property can be a...