We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however involves densities which are very hard to obtain. In this paper we derive exac...

Abstract Two GI/M/1 type Markov chains associated with the queue length are often used in analyzing the discrete time MAP/PH/K queue. The first Markov chain is introduced by tracking service phases for servers; a method we call TPFS. The transition probability matrix of the Markov chain can be constructed in a straightforward manner. The second Markov chain is introduced by counting servers for...

In this study, a mixed $delta$-shock model with discrete-time is defined by combining $delta$-shock and extreme shock models. In this model, a system with multiple states fails in two ways: first, when k interarrival times between two consecutive shocks with magnitude larger than the critical threshold $gamma$ are in $[delta_1, delta _2], delta_1 < delta _2$; and second, when the interarrival t...

A brief introduction to the formulation of various types of stochastic epidemic models is presented based on the well-known deterministic SIS and SIR epidemic models. Three different types of stochastic model formulations are discussed: discrete time Markov chain, continuous time Markov chain and stochastic differential equations. Properties unique to the stochastic models are presented: probab...

Peskun ordering is a partial ordering defined on the space of transition matrices of discrete time Markov chains. If the Markov chains are reversible with respect to a common stationary distribution π, Peskun ordering implies an ordering on the asymptotic variances of the resulting Markov chain Monte Carlo estimators of integrals with respect to π. Peskun ordering is also relevant in the framew...

A variety of phenomena are best described using dynamical models which operate on a discrete state space and in continuous time. Examples include Markov (and semiMarkov) jump processes, continuous-time Bayesian networks, renewal processes and other point processes. These continuous-time, discrete-state models are ideal building blocks for Bayesian models in fields such as systems biology, genet...

In this article, we deal with a class of discrete-time reliability models. The failures are assumed to be generated by an underlying time inhomogeneous Markov chain. The multivariate point process of failures is proved to converge to a Poissontype process when the failures are rare. As a result, we obtain a Compound Poisson approximation of the cumulative number of failures. A rate of convergen...