The hybrid steepest-descent method introduced by Yamada 2001 is an algorithmic solution to the variational inequality problem over the fixed point set of nonlinear mapping and applicable to a broad range of convexly constrained nonlinear inverse problems in real Hilbert spaces. Lehdili and Moudafi 1996 introduced the new prox-Tikhonov regularization method for proximal point algorithm to genera...

In this paper, the methods for use of prior information about multiple operating environments, in improving adaptive filter convergence properties are discussed. More concretely, the gain selection, profiling and scheduling in steepest descent algorithms are treated in detail. Work presented in this paper is an extension of [1]. Two flavors of optimization are discussed: average descent rate op...

GANs excel at learning high dimensional distributions, but they can update generator parameters in directions that do not correspond to the steepest descent direction of the objective. Prominent examples of problematic update directions include those used in both Goodfellow’s original GAN and the WGAN-GP. To formally describe an optimal update direction, we introduce a theoretical framework whi...

The LMS method for linear least squares problems differs from the steepest descent method in that it processes data blocks one-by-one, with intermediate adjustment of the parameter vector under optimization. This mode of operation often leads to faster convergence when far from the eventual limit, and to slower (sublinear) convergence when close to the optimal solution. We embed both LMS and st...

The satissability (SAT) problem is a basic problem in computing theory. Presently, an active area of research on SAT problem is to design eecient optimization algorithms for nding a solution for a satissable CNF formula. A new formulation, the Universal SAT problem model, which transforms the SAT problem on Boolean space into an optimization problem on real space has been developed 31, 35, 34, ...