In this paper we study the long term evolution of a continuous time Markov chain formed by two interacting birth-and-death processes and motivated by modelling interaction between populations. We show transience/recurrence of the Markov chain under fairly general assumptions on transition rates and describe in more detail its asymptotic behaviour in some transient cases. 1 The model and results...

The major goal of this study was to create a continuous time Markov chain (CTMC) models of voltage gating of gap junction (GJ) channels formed of connexin protein. This goal was achieved by using the Piece Linear Aggregate (PLA) formalism to describe the function of GJs and transforming PLA into Markov process. Infinitesimal generator of CTMC was used to automate construction of Markov chain mo...

Continuous-time Markov chains are used extensively to analyze the performance of various computer networks. However, constructing and solving continuous-time Markov chain is a tedious and error-prone procedure, especially when the studied systems are complex. Stochastic Petri nets and the corresponding software packages provide automated generation and solution to continuous-time Markov chains....

Population dynamics are often subject to random independent changes in the environment. For the two strategy stochastic replicator dynamic, we assume that stochastic changes in the environment replace the payoffs and variance. This is modeled by a continuous time Markov chain in a finite atom space. We establish conditions for this dynamic to have an analogous characterization of the long-run b...

The technique presented in this paper allows the automatic construction of a lumped Markov chain for almost symmetrical Stochastic Well-formed Net (SWN) models. The starting point is the Extended Symbolic Reachability Graph (ESRG), which is a reduced representation of a SWN model reachability graph (RG), based on the aggregation of states into classes. These classes may be used as aggregates fo...

Markov jump processes and continuous time Bayesian networks are important classes of continuous time dynamical systems. In this paper, we tackle the problem of inferring unobserved paths in these models by introducing a fast auxiliary variable Gibbs sampler. Our approach is based on the idea of uniformization, and sets up a Markov chain over paths by sampling a finite set of virtual jump times ...

Conditional expected values in Markov chains are solutions to a set of associated backward differential equations, which may be ordinary or partial depending on the number of relevant state variables. This paper investigates the validity of these differential equations by locating the points of non-smoothness of the state-wise conditional expected values, and it presents a numerical method for ...