Proof. It is clear that {Sn 6 t} is completely determined by the history Ft = σ(X(u), u 6 t) until time t. Lemma 1.2. For a homogeneous CTMC, each sojourn time Tn is a continuous memoryless random variable, and the sequence of sojourn times (Tj : j > n) is independent of the past FSn−1 conditioned on Xn−1. Proof. We observe that the sojourn time Tn equals the excess time Y (Sn−1) in state X(Sn−...

Proof. It is clear that {Sn 6 t} is completely determined by the history Ft = σ(X(u), u 6 t) until time t. Lemma 1.2. For a homogeneous CTMC, each sojourn time Tn is a continuous memoryless random variable, and the sequence of sojourn times (Tj : j > n) is independent of the past FSn−1 conditioned on Xn−1. Proof. We observe that the sojourn time Tn equals the excess time Y (Sn−1) in state X(Sn−...

Personal financial decision making plays an important role in modern finance. Decision problems about consumption and insurance are modelled in a continuous-time multi-state Markovian framework. The optimal solution is derived and studied. The model, the problem, and its solution are exemplified by two special cases: In one model the individual takes optimal positions against the risk of dying;...

Abstract Continuous time Markov chains are often used in the literature to model the dynamics of a system with low species count and uncertainty in transitions. In this paper, we investigate three particular algorithms that can be used to numerically simulate continuous time Markov chain models (a stochastic simulation algorithm, explicit and implicit tau-leaping algorithms). To compare these m...

A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We b...