We study the sufficient conditions for the existence of a saddle point of timedependent discrete Markov zero-sum game up to a given stopping time. The stopping time is allowed to take either a finite or an infinite non-negative random variable with its associated objective function being well-defined. The result enables us to show the existence of the saddle points of discrete games constructed...

Hidden Markov Model is a popular statisical method that is used in continious and discrete speech recognition. The probability density function of observation vectors in each state is estimated with discrete density or continious density modeling. The performance (in correct word recognition rate) of continious density is higher than discrete density HMM, but its computation complexity is very ...

A Markov chain is a discrete time Markov process, i.e. the future of the process is determined by the present and independent of the past. Definition 1.2. A Markov process is time-homogeneous if there exists a measurable map P from X into P (X ), the space of probability measures on X , such that P(Xn ∈ A|Xn−1 = a) = P (a;A) for every A ∈ B(X ), almost every a ∈ X , and every n > 0. We call P t...

background: cigarette smoking is one of the most important health‑related risk factors in terms of morbidity and mortality. in this study, we introduced a new method for deriving the transitional probabilities of smoking stages from a cross‑sectional study and simulated a long‑term smoking behavior for adolescents. methods: in this study in 2010, a total of 4853 high school students were random...

For continuous-time Markov chains, we provide criteria for non-ergodicity, non-algebraic ergodicity, non-exponential and non-strong ergodicity. discrete-time ergodicity are given. Our in terms of the existence solutions to inequalities involving Q-matrix (or transition matrix P time-discrete case) chain. Meanwhile, these practical applied some examples, including a special class single birth pr...

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for the embedded Markov chain of semi-Markov processes. Under the assumption that the chain is aperiodic and contains a single class of recurrent states recursive formulae for the variance are obtained which show that the variance growth rate is asymptotically linear in time. Expression...

Markov chains are widely used in the context of performance and reliability evaluation of systems of various nature. Model checking of such chains with respect to a given (branching) temporal logic formula has been proposed for both the discrete [8] and the continuous time setting [1], [3]. In this short paper, we describe the prototype model checker E T MC for discrete and continuous-time Mark...

What follows is a quick survey of the main ingredients in the theory of discrete-time Markov processes. It is a birds' view, rather than the deenitive \state of the art." To maximize accessibility, the nomenclature of mathematical probability is avoided, although rigor is not sacriiced. To compensate, examples (and counterexamples) abound and the bibliography is annotated. Relevance to control ...