An iterative stochastic algorithm to perform maximum a posteriori parameter estimation of hidden Markov models is proposed. It makes the most of the statistical model by introducing an artiicial probability model based on an increasing number of the unobserved Markov chain at each iteration. Under minor regularity assumptions, we provide suucient conditions to ensure global convergence of this ...

Usually, in monitoring a proportion p < /em>, the binary observations are considered independent; however, in many real cases, there is a continuous stream of autocorrelated binary observations in which a two-state Markov chain model is applied with first-order dependence. On the other hand, the Bernoulli CUSUM control chart which is not robust to autocorrelation can be applied two-sided co...

Hidden Markov fields (HMF), which are widely applied in various problems arising in image processing, have recently been generalized to Pairwise Markov Fields (PMF). Although the hidden process is no longer necessarily a Markov one in PMF models, they still allow one to recover it from observed data. We propose in this paper two original methods of parameter estimation in PMF, based on general ...

This paper deals with parameter estimation in pair hidden Markov models (pairHMMs). We first provide a rigorous formalism for these models and discuss possible definitions of likelihoods. The model being biologically motivated, some restrictions with respect to the full parameter space naturally occur. Existence of two different Information divergence rates is established and divergence propert...

The POMDP is considered as a powerful model for planning under uncertainty. However, it is usually impractical to employ a POMDP with exact parameters to model precisely the real-life situations, due to various reasons such as limited data for learning the model, etc. In this paper, assuming that the parameters of POMDPs are imprecise but bounded, we formulate the framework of bounded-parameter...

This work studies the parameter identification problem for the Markov chain choice model of Blanchet, Gallego, and Goyal used in assortment planning. In this model, the product selected by a customer is determined by a Markov chain over the products, where the products in the offered assortment are absorbing states. The underlying parameters of the model were previously shown to be identifiable...

Interval Markov Chains (IMCs) are the base of a classic probabilistic specification theory introduced by Larsen and Jonsson in 1991. They are also a popular abstraction for probabilistic systems. In this paper we study parameter synthesis for a parametric extension of Interval Markov Chains in which the endpoints of intervals may be replaced with parameters. In particular, we propose constructi...

Estimating observability matrices or state sequences is the central component of existing subspace identification methods. In this paper a different approach, in which Markov parameters are first estimated under general input excitation, is proposed. The prominent difference of this approach is that a three-block arrangement of data matrices is used. It is shown that one advantage of this appro...