Least-squares temporal difference learning (LSTD) has been used mainly for improving the data efficiency of the critic in actor-critic (AC). However, convergence analysis of the resulted algorithms is difficult when policy is changing. In this paper, a new AC method is proposed based on LSTD under discount criterion. The method comprises two components as the contribution: (1) LSTD works in an ...

As routing takes place in an entirely distributed system where local routers have no direct access to globally consistent network state information, a routing algorithm has to make uncertain forwarding decisions. As the network state may change, due to failures or new adoptions of networks, routing algorithms have to adapt themselves to the new situation. This network convergence phase should b...

This paper introduces a new method for the digital implementation of controllers designed in continuous time. Through the quantization of its state and input variables the original continuous controller is mapped into a discrete event model within the DEVS formalism framework that can be implemented in a digital device. Under certain conditions on the original continuous control system (CCS), t...

−− A new class of multistep methods for stiff ordinary differential equations is presented. The method is based in the application of estimation functions for the derivatives and the state variables, allowing the transformation of the original system in a purely algebraic system using the solutions of previous steps. From this point of view these methods adopt a semi-implicit scheme. The novelt...

The standard conjugate gradient (CG) method uses orthogonality of the residues to simplify the formulas for the parameters necessary for convergence. In adaptive filtering, the sample-by-sample update of the correlation matrix and the cross-correlation vector causes a loss of the residue orthogonality in a modified online algorithm, which, in turn, results in loss of convergence and an increase...

We develop a primal dual active set with continuation algorithm for solving the l-regularized least-squares problem that frequently arises in compressed sensing. The algorithm couples the the primal dual active set method with a continuation strategy on the regularization parameter. At each inner iteration, it first identifies the active set from both primal and dual variables, and then updates...

This study examines regional convergence in per capita personal income in the US and Canada. We find that the disparity in real per capita income levels across US states (Canadian provinces) has declined, but income levels are not identical. Income levels become more aligned once costs of living are accounted for in relative per capita income series. US states (Canadian provinces) converge at a...

Among many existing algorithms, convergence methods are the most popular means of computing square root and the reciprocal of square root of numbers. An initial approximation is required in these methods. Look up tables (LUT) are employed to produce the initial approximation. In this paper a number of methods are suggested to reduce the size of the look up tables. The precision of the initial a...