Global convergence of an online (stochastic) limited memory version of the Broyden-FletcherGoldfarb-Shanno (BFGS) quasi-Newton method for solving optimization problems with stochastic objectives that arise in large scale machine learning is established. Lower and upper bounds on the Hessian eigenvalues of the sample functions are shown to suffice to guarantee that the curvature approximation ma...

In this paper, the generalized variational inequality with multi-valued mapping (GVI) is considered. To solve the problem, we first establish a global error bound estimation for GVI with the underlying multi-valued mapping being   strict monotone and Holder continuous. Based on this, we propose a new type of method to solve the GVI, and its global convergence is also established. Keywords-GVI...

<span style="color: #333333; font-family: Calibri, sans-serif; font-size: 13.3333px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: justify; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: #ffffff; text-dec...

Data clustering is the process of partitioning a set of data objects into meaning clusters or groups. Due to the vast usage of clustering algorithms in many fields, a lot of research is still going on to find the best and efficient clustering algorithm. K-means is simple and easy to implement, but it suffers from initialization of cluster center and hence trapped in local optimum. In this paper...

This paper shows that for unitary Hessenberg matrices the QR algorithm, with (an exceptional initial-value modification of) the Wilkinson shift, gives global convergence; moreover, the asymptotic rate of convergence is at least cubic, higher than that which can be shown to be quadratic only for Hermitian tridiagonal matrices, under no further assumption. A general mixed shift strategy with glob...

A desire with iterative optimization techniques is that the algorithm reaches the global optimum rather than get stranded at a local optimum value. In this paper, we examine the theoretical and numerical global convergence properties of a certain “gradient free” stochastic approximation algorithm called “SPSA,” that has performed well in complex optimization problems. We establish two theorems ...

‎In this paper‎, ‎we propose a feasible interior-point method for‎ ‎convex quadratic programming over symmetric cones‎. ‎The proposed algorithm relaxes the‎ ‎accuracy requirements in the solution of the Newton equation system‎, ‎by using an inexact Newton direction‎. ‎Furthermore‎, ‎we obtain an‎ ‎acceptable level of error in the inexact algorithm on convex‎ ‎quadratic symmetric cone programmin...

We propose a simple and intuitive method to derive the exact convergence rate of global L2-norm error for strong numerical approximation of stochastic differential equations the result of which has been reported by Hofmann and Müller-Gronbach (2004). We conclude that any strong numerical scheme of order γ > 1/2 has the same optimal convergence rate for this error. The method clearly reveals the...

In this paper, for nonconvex optimization problem with both equality and inequality constrains, we introduce a new augmented Lagrangian function and propose the corresponding multiplier algorithm. The global convergence is established without requiring the boundedness of multiplier sequences. In particular, if the algorithm terminates in finite steps, then we obtain a KKT point of the primal pr...