Semi-Markov processes (SMPs) are real-time stochastic processes in which the time that is spent in a given state before a transition is fired is determined by an arbitrary probability distribution. I Continuous-time Markov chains are a special case of SMPs where all residence-time distributions are exponential. I SMPs have been used extensively to model systems where the time behaviour is not e...

Factored Reinforcement Learning (FRL) is a method to solve Factored Markov Decision Processes when the structure of the transition and reward functions of the problem must be learned. In this paper, we present TeXDYNA, an algorithm that combines the abstraction techniques of Semi-Markov Decision Processes to perform the automatic hierarchical decomposition of the problem with an FRL method. The...

This paper is a survey of results presented in the recent book Gyllenberg and Silvestrov [GS08]. This book is devoted to studies of quasi-stationary phenomena for nonlinearly perturbed stochastic processes and systems. New methods of asymptotic analysis for nonlinearly perturbed stochastic processes based on asymptotic expansions for perturbed renewal equation and recurrence algorithms for cons...