In this article, we propose a method to adapt stepsize parameters used in reinforcement learning for dynamic environments. In general reinforcement learning situations, a stepsize parameter is decreased to zero during learning, because the environment is generally supposed to be noisy but stationary, such that the true expected rewards are fixed. On the other hand, we assume that in the real wo...

Approximate value iteration is used in dynamic programming when we use random observations to estimate the value of being in a state. These observations are smoothed to approximate the expected value function, leading to the problem of choosing a stepsize (the weight given to the most recent observation). A stepsize of 1/n is a common (and provably convergent) choice. However, we prove that it ...

This paper considers the asymptotic stability analysis of both exact and numerical solutions of the following neutral delay differential equation with pantograph delay. ⎧⎨ ⎩ x′(t) +Bx(t) + Cx′(qt) +Dx(qt) = 0, t > 0, x(0) = x0, where B,C,D ∈ Cd×d, q ∈ (0, 1), and B is regular. After transforming the above equation to non-automatic neutral equation with constant delay, we determine sufficient co...

In the numerical solution of ODEs by implicit time-stepping methods, a system of (nonlinear) equations has to be solved each step. It is common practice to use xed-point iterations or, in the stii case, some modiied Newton iteration. The convergence rate of such methods depends on the stepsize. Similarly, a stepsize change may force a refactorization of the iteration matrix in the Newton solver...

The purpose of this manuscript is to derive new convergence results for several subgradient methods for minimizing nonsmooth convex functions with Hölderian growth. The growth condition is satisfied in many applications and includes functions with quadratic growth and functions with weakly sharp minima as special cases. To this end there are four main contributions. First, for a constant and su...

A new approach for acoustic feedback cancellation is presented. The challenge in acoustic feedback cancellation is a strong correlation between the local speech and the loudspeaker signal. Due to this correlation, the convergence rate of adaptive algorithms is limited. Therefore, a novel stepsize control of the adaptive filter is presented. The stepsize control exploits reverberant signal perio...

We propose methods for distributed graph-based multi-task learning that are based on weighted averaging of messages from other machines. Uniform averaging or diminishing stepsize in these methods would yield consensus (single task) learning. We show how simply skewing the averaging weights or controlling the stepsize allows learning different, but related, tasks on the different machines.

This paper is about an efficient implementation of adaptive filtering for echo cancelers. The first objective of this paper is to propose a simplified method of the flexible block Multi-Delay Filter (MDF) algorithm in the time-domain. Then, we will derive a new method for the stepsize adaptation coefficient. The second objective is about the realization of a Block Proportionate Normalized Least...

This paper investigates the performance of two explicit pseudo two-step Runge-Kutta methods of order 5 and 8 for rst-order nonstii ODEs on a parallel shared memory computer. For expensive right hand sides the parallel implementation gives a speedup of 3{4 with respect to the sequential one. Furthermore we compare the codes with the two eecient nonstii codes DOPRI5 and DOP853. For problems, wher...