This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study the interrelation of reactive processes and Markov chains in this setting, and introduce the algebr...

This paper will explore the basics of discrete-time Markov chains used to prove the Ergodic Theorem. Definitions and basic theorems will allow us to prove the Ergodic Theorem without any prior knowledge of Markov chains, although some knowledge about Markov chains will allow the reader better insight about the intuitions behind the provided theorems. Even for those familiar with Markov chains, ...

Recent years have seen a considerable increase of attention devoted to Poisson’s equation for Markov chains, which now has attained a central place in Markov chain theory, due to the extensive list of areas where Poisson’s equation pops up: perturbation analysis, Markov decision processes, limit theorems of Markov chains, etc. all find natural expression when viewed from the vantage point of Po...

This paper presents a simulation study on Fuzzy Markov chains to identify some characteristics about their behavior, based on matrix analysis. Through experimental evidence it is observed that most of fuzzy Markov chains does not have an ergodic behavior. So, several sizes of Markov chains are simulated and some statistics are collected. Two methods for obtaining the Stationary Distribution of ...

Markov chains have been used to model web usages and served as the basis for statistical testing, performance evaluation, and reliability analysis. However, most of such applications of Markov chains were carried out without answering the question: “Can web usage be accurately modeled by Markov chains?” In this paper, we propose a set of tests, which are easy to perform based on information abo...