Tuning stepsize between convergence rate and steady state error level or stability is a problem in some subspace tracking schemes. Methods in DPM and OJA class may show sparks in their steady state error sometimes, even with a rather small stepsize. By a study on the schemes’ updating formula, it is found that the update only happens in a specific plane but not all the subspace basis. Through a...

We consider the emphatic temporal-difference (TD) algorithm, ETD(λ), for learning the value functions of stationary policies in a discounted, finite state and action Markov decision process. The ETD(λ) algorithm was recently proposed by Sutton, Mahmood, and White [47] to solve a long-standing divergence problem of the standard TD algorithm when it is applied to off-policy training, where data f...

The DI methods for directly solving a system ofa general higher order ODEs are discussed. The convergence of the constant stepsize and constant order formulation of the DI methods is proven first before the convergencefor the variable order and stepsize case.

The most widely used algorithm for training multiplayer feedforward networks, Error BackPropagation (EBP), is an iterative gradient descend algorithm by nature. Variable stepsize is the key to fast convergence of BP networks. A new optimal stepsize algorithm is proposed for accelerating the training process. It modifies the objective function to reduce the computational complexity of the Jacobi...

Conditions on Runge-Kutta algorithms can be obtained which ensure smooth stepsize selection when stability of the algorithm is restricting the stepsize. Some recently derived results are shown to hold for a more general test problem.

The definition of the standard derivative operator is extended from integer steps to arbitrary stepsize. The classical, nonrelativistic Hamiltonian is quantized, using these new fractional operators. The resulting Schroedinger type equation generates free particle solutions, which are confined in space. The angular momentum eigenvalues are calculated algebraically. It is shown, that the charmon...

In this paper, some privacy-preserving features for distributed subgradient optimization algorithms are considered. Most of the existing distributed algorithms focus mainly on the algorithm design and convergence analysis, but not the protection of agents' privacy. Privacy is becoming an increasingly important issue in applications involving sensitive information. In this paper, we first show t...

We consider an incremental gradient method with momentum term for minimizing the sum of continuously differentiable functions. This method uses a new adaptive stepsize rule that decreases the stepsize whenever sufficient progress is not made. We show that if the gradients of the functions are bounded and Lipschitz continuous over a certain level set, then every cluster point of the iterates gen...

In this article, we propose a method to adapt stepsize parameters used in reinforcement learning for dynamic environments. In general reinforcement learning situations, a stepsize parameter is decreased to zero during learning, because the environment is generally supposed to be noisy but stationary, such that the true expected rewards are fixed. On the other hand, we assume that in the real wo...