This paper mainly analyzes the applications of the Generator matrices in a Continuous Time Markov Chain (CTMC). Hidden Markov models [HMMs] together with related probabilistic models such as Stochastic Context-Free Grammars [SCFGs] are the basis of many algorithms for the analysis of biological sequences. Combined with the continuous-time Markov chain theory of likelihood based phylogeny, stoch...

In this study we model the daily rainfall occurrence using Markov Chain Analogue Yearmodel (MCAYM) and the intensity or amount of daily rainfall using three different probability distributions; gamma, exponential and mixed exponential distributions. Combining the occurrence and intensity model we obtain Markov Chain Analogue Year gamma model (MCAYGM), Markov Chain Analogue Year exponentia...

The problem we investigate deals with the optimal assignment of resources to the activities of a stochastic project network. We seek to minimize the expected cost of the project include sum of resource utilization costs and lateness costs. We assume that the work content required by the activities follows an exponential distribution. The decision variables of the model are the allocated resourc...

 In this paper we study the relative entropy rate between a homogeneous Markov chain and a hidden Markov chain defined by observing the output of a discrete stochastic channel whose input is the finite state space homogeneous stationary Markov chain. For this purpose, we obtain the relative entropy between two finite subsequences of above mentioned chains with the help of the definition of...

Jump Markov linear systems (JMLSs) are linear systems whose parameters evolve with time according to a finite state Markov chain. Given a set of observations, our aim is to estimate the states of the finite state Markov chain and the continuous (in space) states of the linear system. In this paper, we present original deterministic and stochastic iterative algorithms for optimal state estimatio...

This paper surveys and relates the basic concepts of process algebra and the modelling of continuous time Markov chains. It provides basic introductions to both fields, where we also study the Markov chains from an algebraic perspective, viz. that of Markov chain algebra. We then proceed to study the interrelation of reactive processes and Markov chains in this setting, and introduce the algebr...

Numerical observations on a Markov chain and on the continuous Markov process performed by a granular tracer show that the " usual " fluctuation relation for a given observable is not verified for finite (but arbitrarily large) times. This suggests that some terms which are usually expected to be negligible, i.e. " border terms " dependent only on initial and final states, in fact cannot be neg...

In this note we explore an aspect of the relationship between the notion of equilibrium of a continuous time Markov chain (CTMC) and that of the traditional concept of termination in rewriting systems. Unlike in deterministic dynamical systems, a Markov chain equilibrium is not a definite state, but rather a probability over the state space which is invariant under the Markov semigroup, and sat...