We consider LSTD(λ), the least-squares temporal-difference algorithm with eligibility traces algorithm proposed by Boyan (2002). It computes a linear approximation of the value function of a fixed policy in a large Markov Decision Process. Under a β-mixing assumption, we derive, for any value of λ ∈ (0, 1), a high-probability estimate of the rate of convergence of this algorithm to its limit. W...

The true online TD(λ) algorithm has recently been proposed (van Seijen and Sutton, 2014) as a universal replacement for the popular TD(λ) algorithm, in temporal-difference learning and reinforcement learning. True online TD(λ) has better theoretical properties than conventional TD(λ), and the expectation is that it also results in faster learning. In this paper, we put this hypothesis to the te...

TD(λ) is a core algorithm of modern reinforcement learning. Its appeal comes from its equivalence to a clear and conceptually simple forward view, and the fact that it can be implemented online in an inexpensive manner. However, the equivalence between TD(λ) and the forward view is exact only for the off-line version of the algorithm (in which updates are made only at the end of each episode). ...

Motivated by parallel optimization, we experiment EDA-like adaptation-rules in the case of λ large. The rule we use, essentially based on estimation of multivariate normal algorithm, is (i) compliant with all families of distributions for which a density estimation algorithm exists (ii) simple (iii) parameter-free (iv) better than current rules in this framework of λ large. The speed-up as a fu...

Fabrication of an integrated circuit with smaller area, besides reducing the cost of manufacturing, usually causes a reduction in the power dissipation and propagation delay. Using the static CMOS technology to fabricate a circuit that realizes a specific logic function and occupies a minimum space, it must be implemented with continuous diffusion runs. Therefore, at the design stage, an Euleri...

Fabrication of an integrated circuit with smaller area, besides reducing the cost of manufacturing, usually causes a reduction in the power dissipation and propagation delay. Using the static CMOS technology to fabricate a circuit that realizes a specific logic function and occupies a minimum space, it must be implemented with continuous diffusion runs. Therefore, at the design stage, an Euleri...

In this paper, we give necessary and sufficient conditions for a set of Jordan blocks to correspond to the peripheral spectrum of a nonnegative matrix. For each eigenvalue, λ, the λ-level characteristic (with respect to the spectral radius) is defined. The necessary and sufficient conditions include a requirement that the λ-level characteristic is majorized by the λ-height characteristic. An al...

In practice, obtaining the global optimum for the economic dispatch {bf (ED)}problem with ramp rate limits and prohibited operating zones is presents difficulties. This paper presents a new andefficient method for solving the economic dispatch problem with non-smooth cost functions using aFuzzy Adaptive Genetic Algorithm (FAGA). The proposed algorithm  deals  with the issue ofcontrolling the ex...