We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero. Let (Xn) be a Markov chain with finite state space S and transition matrix P (p) and let (Yn) be the Hidden Markov chain observed by passing (Xn...

We consider weak lumpability of denumerable discrete or continuous time Markov chains. Firstly, we are concerned with irreducible recurrent positive and R-positive Markov chains evolving in discrete time. We study the properties of the set of all initial distributions of the starting chain leading to an aggregated homogeneous Markov chain with respect to a partition of the state space. In parti...

As mentioned in the paper Markov Chain Monte Carlo (MCMC) is a technique used to obtain samples from probability distributions known only up to a normalising factor. Given that p(~ θ) is a nonnegative integrable function, the Metropolis Hastings algorithm will provide a sequence of samples (also known as a chain) whose equilibrium distribution is proportional to p(~ θ) using only evaluations of...

A positive recurrent, aperiodic Markov chain is said to be long range dependent (LRD) when the indicator function of a particular state is LRD. This happens if and only if the return time distribution for that state has infinite variance. We investigate the question of whether other instantaneous functions of the Markov chain also inherit this property. We provide conditions under which the fun...

Introduction. We consider Markov chains with stationary transition probabilities and state space S = 0, 1, 2, •• -, in both discrete and continuous time. As in [l ] the transition probabilities are denoted by pi}1 (indiscrete time) and pa(t) (in continuous time). We assume that the chains considered are irreducible and recurrent, and in addition that discrete time chains are aperiodic and that ...

We investigate the importance sampling (IS) simulation for the sample average of an output sequence from an irreducible Markov chain. The optimal Markov chain used in simulation is known to be a twisted Markov chain, however, the proofs in [2], [3] are very complicated and do not give us a good perspective. We give a simple and natural proof for the optimality of the simulation Markov chain in ...

In this article, relying on Foster–Lyapunov drift conditions, we establish subexponential upper and lower bounds the rate of convergence in $$\text {L}^p$$ -Wasserstein distance for a class irreducible aperiodic Markov processes. We further discuss these results context Lévy-type lack irreducibility and/or aperiodicity properties, obtain exponential ergodicity Itô processes under an asymptotic ...

The purpose of this article is to present results concerning the sensitivity of the stationary probabilities for a n-state, time-homogeneous, irreducible Markov chain in terms of the mean first passage times in the chain.

The purpose of this paper is to review and compare the existing perturbation bounds for the stationary distribution of a finite, irreducible, homogeneous Markov chain.