The EM algorithm, e.g. as Baum-Welch reestimation, is an important tool for parameter estimation in discrete-time Hidden Markov Models. We present a direct reestimation of rate constants for applications in which the underlying Markov process is continuous in time. Previous estimation of discrete-time transition probabilities is not necessary.

The EM algorithm, e.g., the Baum–Welch re-estimation, is an important tool for parameter estimation in discrete-time hidden Markov models. We present a direct re-estimation of rate constants for applications in which the underlying Markov process is continuous in time. Previous estimation of discrete-time transition probabilities is not necessary.

Average cost Markov decision chains with discrete time parameter are considered. The cost function is unbounded and satisfies an additional condition which frequently holds in applications. Also, we assume that there exists a single stationary policy for which the corresponding Markov chain is irreducible and ergodic with finite average cost. Within this framework, the existence of an average c...

It was recently shown that there exists a family Z2 Markov random fields which are K but are not isomorphic to Bernoulli shifts [4]. In this paper we show that most distinct members of this family are not isomorphic. This implies that there is a two parameter family of Z2 Markov random fields of the same entropy, no two of which are isomorphic.

The present paper is concerned with the eetlmatlon of the transition distributions of a Markov renewal process with finitely many states, which extends and unifies nome aspects of the results in the special cases of discrete and continuous parameter Markov chains. A natural estimator of the transition distribution." is defined and shown to be consistent. Limiting distributions of this estimator...