In this paper, the ambiguity of nite state irreducible Markov chain trajectories is reminded and is obtained for two state Markov chain. I give an applicable example of this concept in President election

This paper considers the problem of veriication of proba-bilistic systems, for both the discrete-and real-time cases. Probabilistic systems are modeled as Markov chains, concurrent Markov chains (i.e. MCs with non-determinism) or, in the real-time case, generalized semi-Markov processes (Markovian processes with time delays on events). Discrete-time speciications are described in the logics PTL...

We consider the value 1 problem for probabilistic automata over finite words. This problem is known to be undecidable. However, different algorithms have been proposed to partially solve it. The aim of this paper is to prove that one such algorithm, called the Markov Monoid algorithm, is optimal. To this end, we develop a profinite theory for probabilistic automata. This new framework gives a t...

We marry continuous time Markov decision processes (CTMDPs) with stochastic timed automata into a model with joint expressive power. This extension is very natural, as the two original models already share exponentially distributed sojourn times in locations. It enriches CTMDPs with timing constraints, or symmetrically, stochastic timed automata with one conscious player. Our model maintains th...

The implementation of China’s ex situ poverty alleviation and relocation project has alleviated the further deterioration ecological environment in area. It can create favorable conditions for management problems such as natural restoration rocky desertification soil erosion. Panzhou City, Guizhou Province, is one key areas projects 13th Five-Year Plan National Economic Social Development. typi...

Abstract: This paper describes a model for incorporating both spatial interactions and succession dynamics within a MOSAIC forest landscape model. MOSAIC uses semi-Markov processes to model the succession dynamics of a forest landscape. A landscape is partitioned into cells and the parameters defining the semiMarkov process associated with each cell are derived from individual tree-based models...

The cellular learning automata, which is a combination of cellular automata, and learning automata, is a new recently introduced model. This model is superior to cellular automata because of its ability to learn and is also superior to a single learning automaton because it is a collection of learning automata which can interact with each other. The basic idea of cellular learning automata, whi...

Markov games, as the generalization of Markov decision processes to the multi agent case, have long been used for modeling multi-agent systems. In this paper, several learning automata based multi-agent system algorithms for finding optimal policies in fully-cooperative Markov Games are proposed. In the proposed algorithms, Markov problem is described as a directed graph in which the nodes are ...

This chapter surveys some basic algorithms for analyzing Markov chains (MCs) and Markov decision processes (MDPs), and discusses their computational complexity. We focus on discrete-time processes, and we consider both finite-state models as well as countably infinite-state models that are finitely-presented. The analyses we will primarily focus on are hitting (reachability) probabilities and ω...