We consider the stochastic shortest path problem, a classical finite-state Markovian decision problem with a termination state, and we propose new convergent Q-learning algorithms that combine elements of policy iteration and classical Q-learning/value iteration. These algorithms are related to the ones introduced by the authors for discounted problems in Bertsekas and Yu (Math. Oper. Res. 37(1...

A preconditioned conjugate gradient method was implemented into an iteration on a program for data estimation of breeding values, and its convergence characteristics were studied. An algorithm was used as a reference in which one fixed effect was solved by Gauss-Seidel method, and other effects were solved by a second-order Jacobi method. Implementation of the preconditioned conjugate gradient ...

The classical Newton iteration method for matrices can be modified into an efficient algorithm when structured matrices are involved. The difficulty, however, is the importance of the choice of the starting matrix. In this paper, we propose a new initial iteration step which makes the choice of the starting matrix less critical. The validity of the approach is illustrated by numerical experiments.

We prove that under certain basic regularity conditions, a random iteration of logistic maps converges to a random point attractor when the Lyapunov exponent is negative, and does not converge to a point when the Lyapunov exponent is positive.

Value iteration is a commonly used and em­ pirically competitive method in solving many Markov decision process problems. However, it is known that value iteration has only pseudo­ polynomial complexity in general. We estab­ lish a somewhat surprising polynomial bound for value iteration on deterministic Markov decision (DMDP) problems. We show that the basic value iteration procedure converges...

Value iteration is a commonly used and empirically competitive method in solving many Markov decision process problems. However, it is known that value iteration has only pseudopolynomial complexity in general. We establish a somewhat surprising polynomial bound for value iteration on deterministic Markov decision (DMDP) problems. We show that the basic value iteration procedure converges to th...

Markov Decision Processes (MDPs) are used to model both non-deterministic and probabilistic systems. Probabilistic model checking is an approach for verifying quantitative properties of probabilistic systems that are modeled by MDPs. Value and Policy Iteration and modified version of them are well-known approaches for computing a wide range of probabilistic properties. This paper tries to impro...

Let E be a real Banach Space and K a nonempty closed convex (not necessarily bounded) subset of E. Iterative methods for the approximation of fixed points of asymptotically demicontractive mappings T : K → K are constructed using the more general modified Mann and Ishikawa iteration methods with errors. Our results show that a recent result of Osilike [3] (which is itself a generalization of a ...

In this paper we propose a novel algorithm, factored value iteration (FVI), for the approximate solution of factored Markov decision processes (fMDPs). The traditional approximate value iteration algorithm is modified in two ways. For one, the least-squares projection operator is modified so that it does not increase max-norm, and thus preserves convergence. The other modification is that we un...