Bounds on convergence rates for Markov chains are a very widely-studied topic, motivated largely by applications to Markov chain Monte Carlo algorithms. For Markov chains on finite state spaces, previous authors have obtained a number of very useful bounds, including those which involve choices of paths. Unfortunately, many Markov chains which arise in practice are not finite. In this paper, we...

In this research paper, weighted / unweighted, directed / undirected graphs are associated with interesting Discrete Time Markov Chains (DTMCs) as well as Continuous Time Markov Chains ( CTMCs ). The equilibrium / transient behavior of such Markov chains is studied. Also entropy dynamics ( Shannon entropy ) of certain structured Markov chains is investigated. Finally certain structured graphs a...

A parallel composition is defined for Markov reward chains with stochastic discontinuity, and with fast and silent transitions. In this setting, compositionality with respect to the relevant aggregation preorders is established. For Markov reward chains with fast transitions the preorders are τ -lumping and τ -reduction. Discontinuous Markov reward chains are ‘limits’ of Markov reward chains wi...

In this paper at rst, a history of mathematical models is given.Next, some basic information about random variables, stochastic processesand Markov chains is introduced. As follows, the entropy for a discrete timeMarkov process is mentioned. After that, the entropy for SIS stochastic modelsis computed, and it is proved that an epidemic will be disappeared after a longtime.