We consider the stochastic shortest path problem, a classical finite-state Markovian decision problem with a termination state, and we propose new convergent Q-learning algorithms that combine elements of policy iteration and classical Q-learning/value iteration. These algorithms are related to the ones introduced by the authors for discounted problems in Bertsekas and Yu (Math. Oper. Res. 37(1...

The classical Newton iteration method for matrices can be modified into an efficient algorithm when structured matrices are involved. The difficulty, however, is the importance of the choice of the starting matrix. In this paper, we propose a new initial iteration step which makes the choice of the starting matrix less critical. The validity of the approach is illustrated by numerical experiments.

The author considered both the direct effect and the moderator effect of role salience in the stress-strain relationship. In contrast to previous studies that have examined the effects of salience on well-being within specific social roles, the present study focused on the work-family interface. From a sample of 147 employed English women with children, the present results of the regression ana...

Markov Decision Processes (MDPs) are used to model both non-deterministic and probabilistic systems. Probabilistic model checking is an approach for verifying quantitative properties of probabilistic systems that are modeled by MDPs. Value and Policy Iteration and modified version of them are well-known approaches for computing a wide range of probabilistic properties. This paper tries to impro...

Let E be a real Banach Space and K a nonempty closed convex (not necessarily bounded) subset of E. Iterative methods for the approximation of fixed points of asymptotically demicontractive mappings T : K → K are constructed using the more general modified Mann and Ishikawa iteration methods with errors. Our results show that a recent result of Osilike [3] (which is itself a generalization of a ...

In this paper we propose a novel algorithm, factored value iteration (FVI), for the approximate solution of factored Markov decision processes (fMDPs). The traditional approximate value iteration algorithm is modified in two ways. For one, the least-squares projection operator is modified so that it does not increase max-norm, and thus preserves convergence. The other modification is that we un...

The input motion for seismic analysis of building structures is generally specified in design codes in terms of response spectrum. Therefore, real or simulated time histories for ground motions whose response spectrum matches approximately the design spectrum is desired. Among the various techniques currently available for generating spectrum compatible time histories it is possible to distingu...

We present a stochastic planner based on Markov Decision Processes (MDPs) that participates to the probabilistic planning track of the 2006 International Planning Competition. The planner transforms the PPDDL problems into factored MDPs that are then solved with a structured modified value iteration algorithm based on the safest stochastic path computation from the initial states to the goal st...

This paper deals with a modification of the Hill cipher. In this, we have introduced interweaving in each step of the iteration. The interweaving of the resulting plaintext, at each stage of the iteration, and the multiplication with the key matrix leads to confusion and diffusion. From the cryptanalysis performed in this investigation, we have found that the cipher is a strong one.