It is known that the second dominant eigenvalue of a matrix determines the convergence rate of the power method. Though ineffective for general eigenvalue computation, the power method has been of practical usage for computing the stationary distribution of a stochastic matrix. For a Markov chain with memory m, the transition “matrix" becomes an order-m tensor. Under suitable assumptions, the s...

Consider a nite state space Ω and a transition kernel P : Ω ×Ω→ [0, 1] such that for every x ∈ Ω, ∑ y∈Ω P(x , y) 1. The Markov chain corresponding to the kernel P is the sequence of random variables {X0 ,X1 ,X2 , . . .} such that for every t > 0, we have [Xt+1 y | Xt x] P(x , y). Note that we also have to specify a distribution for the initial state X0. Corresponding to every such process, one...

Consider a parametric statistical model P (dx|θ) and an improper prior distribution ν(dθ) that together yield a (proper) formal posterior distribution Q(dθ|x). The prior is called strongly admissible if the generalized Bayes estimator of every bounded function of θ is admissible under squared error loss. Eaton [Ann. Statist. 20 (1992) 1147–1179] has shown that a sufficient condition for strong ...

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for the embedded Markov chain of semi-Markov processes. Under the assumption that the chain is aperiodic and contains a single class of recurrent states recursive formulae for the variance are obtained which show that the variance growth rate is asymptotically linear in time. Expression...

This article presents a new numerical method for approximately computing the ergodic projector of a finite aperiodic Markov chain. Our approach requires neither structural information on the chain, such as, the identification of ergodic classes, transient states, or qualitative information, such as, whether the chain is nearly decomposable or not. The theoretical deduction of the new method is ...

It is our pleasure to congratulate the authors (hereafter DKSC) on an interesting paper that was a delight to read. While DKSC provide a remarkable collection of connections between different representations of the Markov chains in their paper, we will focus on the “running time analysis” portion. This is a familiar problem to statisticians; given a target population, how can we obtain a repres...

Proof of Lemma A.3. Let Assumptions 3.2 and 3.3 hold. Assume that firms follow a common oblivious strategy µ ∈ ˜ M, the expected entry rate is λ ∈ ˜ Λ, and the expected time that each firm spends in the industry is finite. Let {Z x : x ∈ N} be a sequence of independent Poisson random variables with means {˜s µ,λ (x) : x ∈ N}, and let Z be a Poisson random variable with mean x∈N˜s µ,λ (x). Then,...

Markov chain Monte Carlo, or MCMC, is a way to sample probability distributions that cannot be sampled practically using direct samplers. Most complex probability distributions in more than a few variables are are sampled in this way. For us, a stationary Markov chain is a random sequence X1, X2, . . ., where Xk+1 = M(Xk, ξk), where M(x, ξ) is a fixed function and the inputs ξ are i.i.d. random...