The present paper investigates the embedding problem for time-homogeneous Markov chains. A discretetime Markov chain with time unit 1 is embeddable in case there exists a compatible Markov chain regarding time unit 1 m (with m ∈ N,m ≥ 2). An embeddable Markov chain has a transition matrix for which there exists an m-th root that is a probability matrix. The present paper examines the embedding ...

In this paper, we study a model of quantum Markov chains that is a quantum analogue of Markov chains and is obtained by replacing probabilities in transition matrices with quantum operations. We show that this model is very suited to describe hybrid systems that consist of a quantum component and a classical one, although it has the same expressive power as another quantum Markov model proposed...

Exact and ordinary lumpability in nite Markov chains is considered. Both concepts naturally deene an aggregation of the Markov chain yielding an aggregated chain that allows the exact determination of several stationary and transient results for the original chain. We show which quantities can be determined without an error from the aggregated process and describe methods to calculate bounds on...

We introduce a form of Rao{Blackwellization for Markov chains which uses the transition distribution for conditioning. We show that for reversible Markov chains, this form of Rao{Blackwellization always reduces the asymptotic variance, and derive two explicit forms of the variance reduction obtained through repeated Rao{Blackwellization. The result applies to many Markov chain Monte Carlo metho...

We study the asymptotic behavior of kernel estimators of asymptotic variances (or long-run variances) for a class of adaptive Markov chains. The convergence is studied both in L and almost surely. The results apply to Markov chains as well and improve on the existing literature by imposing weaker conditions. We illustrate the results with applications to the GARCH(1, 1) Markov model and to an a...

[Tip: Study the MC, QT, and Little's law lectures together: CTMC (MC lecture), M/M/1 queue (QT lecture), Little's law lecture (when deriving the mean response time from mean number of customers), DTMC (MC lecture), M/M/1 queue derivation using DTMC analysis, derive distribution of response time in M/M/1 queue (QT lecture), relation between Markov property and mem-oryless property (MC lecture), ...

To establish lists of words with unexpected frequencies in long sequences, for instance in a molecular biology context, one needs to quantify the exceptionality of families of word frequencies in random sequences. To this aim, we study large deviation probabilities of multidimensional word counts for Markov and hidden Markov models. More specifically, we compute local Edgeworth expansions of ar...

Bounds on convergence rates for Markov chains are a very widely-studied topic, motivated largely by applications to Markov chain Monte Carlo algorithms. For Markov chains on finite state spaces, previous authors have obtained a number of very useful bounds, including those which involve choices of paths. Unfortunately, many Markov chains which arise in practice are not finite. In this paper, we...

We consider the stochastic model of water pollution, which mathematically can be written with a stochastic partial differential equation driven by Poisson measure noise. We use a stochastic particle Markov chain method to produce an implementable approximate solution. Our main result is the annealed law of large numbers establishing convergence in probability of our Markov chains to the solutio...