Two new PRP conjugate Algorithms are proposed in this paper based on two modified PRP conjugate gradient methods: the first algorithm is proposed for solving unconstrained optimization problems, and the second algorithm is proposed for solving nonlinear equations. The first method contains two aspects of information: function value and gradient value. The two methods both possess some good prop...

A Projected-Gradient Underdetermined Newton-like algorithm will be introduced for finding a solution of a Horizontal Nonlinear Complementarity Problem (HNCP) corresponding to a feasible solution of a Mathematical Programming Problem with Complementarity Constraints (MPCC). The algorithm employs a combination of Interior-Point Newton-like and Projected-Gradient directions with a line-search proc...

An important variation of preconditioned conjugate gradient algorithms is inexact precon-ditioner implemented with inner-outer iterations 5], where the preconditioner is solved by an inner iteration to a prescribed precision. In this paper, we formulate an inexact preconditioned conjugate gradient algorithm for a symmetric positive deenite system and analyze its convergence property. We establi...

Stochastic variance reduction algorithms have recently become popular for minimizing the average of a large but finite number of loss functions. In this paper, we propose a novel Riemannian extension of the Euclidean stochastic variance reduced gradient algorithm (R-SVRG) to a manifold search space. The key challenges of averaging, adding, and subtracting multiple gradients are addressed with r...

Agents controlled by a swarm algorithm interact with each other so that they have collective capabilities that a single agent does not have. The bio-inspired swarm-algorithm “BEECLUST” has the aim to aggregate a swarm at the global optimum even if there are several local optima (of the same type) present. But what about gradients produced of different stimulus types? In this paper, we present t...

In theory, the successive gradients generated by the conjugate gradient method applied to a quadratic should be orthogonal. However, for some ill-conditioned problems, orthogonality is quickly lost due to rounding errors, and convergence is much slower than expected. A limited memory version of the nonlinear conjugate gradient method is developed. The memory is used to both detect the loss of o...

This paper proposes a novel optimization algorithm inspired by water flowing and shaking behaviors in a vessel. Water drops in our algorithm flow to the gradient descent direction and are sometimes shaken for getting out of local optimum areas when most water drops fall in local optimum areas. These flowing and shaking operations allow our algorithm to quickly approach to the global optimum wit...

Particle swarm and simulated annealing optimization algorithms proved to be valid in finding a global optimum in the bound constrained optimization context. However, their original versions can only detect one global optimum even if the problem has more than one solution. In this paper we propose modifications to both algorithms. In the particle swarm optimization algorithm we introduce gradien...

Recently, the existed proximal gradient algorithms had been used to solve non-smooth convex optimization problems. As a special nonsmooth convex problem, the singly linearly constrained quadratic programs with box constraints appear in a wide range of applications. Hence, we propose an accelerated proximal gradient algorithm for singly linearly constrained quadratic programs with box constraint...

global optimization methods play an important role to solve many real-world problems. flower pollination algorithm (fp) is a new nature-inspired algorithm, based on the characteristics of flowering plants. in this paper, a new hybrid optimization method called hybrid flower pollination algorithm (fppso) is proposed. the method combines the standard flower pollination algorithm (fp) with the par...