Markov chain models had proved to be useful tools in many fields, such as physics, chemistry, information sciences, economics, finances, mathematical biology, social sciences, and statistics for analyzing data. A discrete time Markov chain is often used as a statistical model from a random physical process to fit the observed data. A time-homogeneous Markov chain is a process that each transiti...

A novel approach for obtaining the response time in a discrete time tandem queue with blocking, using matrix analytic methods, is presented. The approach sets up the appropriate Markov chain based on the age of the leading customer in the first queue, together with other auxiliary variables. By this approach the response time is obtained with only minor additional effort after obtaining the sta...

A hidden Markov process (HMP) is a discrete-time finite-state homogeneous Markov chain observed through a discrete-time memoryless invariant channel. The non-asymptotic equipartition property (NEP) is a bound on the probability of the sample entropy deviating from the entropy rate of a stochastic process, so it can be viewed as a refinement of Shannon-McMillan-Breiman theorem. In this report, w...

The statistical literature discusses different types of Markov properties for chain graphs that lead to four possible classes of chain graph Markov models. The different models are rather well understood when the observations are continuous and multivariate normal, and it is also known that one model class, referred to as models of LWF (Lauritzen–Wermuth–Frydenberg) or block concentration type,...

We describe some old and new methods for coupling the flow of a discrete time Markov chain, provided its transition function is known. Examples including Hastings-Metropolis and Gibbs samplers are given.

Consider a discrete-time Markov chain (Xn : n > 0) ∼ Markov(λ, P ) with countable state space S. In the last lecture, we proved the following theorem for aperiodic Markov chains. Theorem (Aperiodic case). Assume P is irreducible and positive recurrent with unique invariant measure π. If, in addition, P is aperiodic, then P(Xn = j)→ πj as n→∞ for all j ∈ S. In particular, p (n) ij → πj as n→∞ fo...

To estimate the premium an investor should expect from extended hedge fund lockups, Derman et al. (2009) proposed a three-state discrete-time Markov Chain to model the state of a hedge fund, allowing the state to change randomly among the states “good,” “sick” and “dead” every year. The lockup premium measures the consequence of being stuck with a sick fund. To be more realistic, we propose an ...

We address the problem of finding a natural continuous time Markov type process—in open populations—that best captures information provided by an chain in discrete which is usually sole possible observation from data. Given chain, we single out two main approaches: In first one, consider calibration procedure process using transition matrix and show that, when embeddable has optimal solutions. ...

We show that a large class of discrete choice models which contain the Markov chain model introduced by Blanchet, Gallego, and Goyal (2013) belong to the class of discrete choice models based on random utility. © 2016 Elsevier B.V. All rights reserved.