In this paper three methods for on-line identification of first markov parameter at linear multivariable plants are presented. In these methods input-output data are used far the on-line identification of the first markov parameter.

Nowadays, Hidden Markov models are extensively utilized for modeling stochastic processes. These models help researchers establish and implement the desired theoretical foundations using Markov algorithms such as Forward one. however, Using Stability hypothesis and the mean statistic for determining the values of Markov functions on unstable statistical data set has led to a significant reducti...

In this paper, we introduce the notion of a bounded parameter Markov decision process as a generalization of the traditional exact MDP. A bounded parameter MDP is a set of exact MDPs speciied by giving upper and lower bounds on transition probabilities and rewards (all the MDPs in the set share the same state and action space). Bounded parameter MDPs can be used to represent variation or uncert...

nowadays, hidden markov models are extensively utilized for modeling stochastic processes. these models help researchers establish and implement the desired theoretical foundations using markov algorithms such as forward one. however, using stability hypothesis and the mean statistic for determining the values of markov functions on unstable statistical data set has led to a significant reducti...

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...