The TemporalMobile Stochastic Logic (MOSL) has been introduced in previous work by the authors for formulating properties of systems specified in STOKLAIM, a Markovian extension of KLAIM. The main purpose of MOSL is to address key functional aspects of global computing such as distribution awareness, mobility, and security and their integration with performance and dependability guarantees. In ...

growing amount of information on biological sequences has made application of statistical approaches necessary for modeling and estimation of their functions. in this paper, sensitivity and specificity of the first and second markov chains for prediction of genes was evaluated using the complete double stranded  dna virus. there were two approaches for prediction of each markov model parameter,...

The goal of dialogue management in a spoken dialogue system is to take actions based on observations and inferred beliefs. To ensure that the actions optimize the performance or robustness of the system, researchers have turned to reinforcement learning methods to learn policies for action selection. To derive an optimal policy from data, the dynamics of the system is often represented as a Mar...

Abstract: Stochastic models such as mixture models, graphical models, Markov random fields and hidden Markov models have key role in probabilistic data analysis. In this paper, we used Gaussian mixture model to the pixels of an image. The parameters of the model were estimated by EM-algorithm.   In addition pixel labeling corresponded to each pixel of true image was made by Bayes rule. In fact,...

Invited Paper In this paper, we discuss the role of modeling in the design and validation of life-critical, real-time systems. The basics of Markov, Markov reward, and stochastic reward net models are covered. An example of a nuclear power plant cooling system is developed in detail. Multilevel models, model calibration, and model validation are also discussed. I. INTRODUCTION Modem industrial ...

We present an approach to the detection of global environmental regime changes by a mobile robot performing a task. The approach is based on the use of augmented Markov models (AMMs), a variation of semi-Markov process. We have developed an algorithm that constructs AMMs on-line and in real-time with little computational and space overhead. AMMs are a general tool for capturing the interaction ...

Markov control algorithms that perform smooth, non-greedy updates of the policy have been shown to be very general and versatile, with policy gradient and Expectation Maximisation algorithms being particularly popular. For these algorithms, marginal inference of the reward weighted trajectory distribution is required to perform policy updates. We discuss a new exact inference algorithm for thes...

Action delays degrade the performance of reinforcement learning in many real-world systems. This paper proposes a formal definition delay-aware Markov Decision Process and proves it can be transformed into standard MDP with augmented states using reward process. We develop model-based framework that incorporate multi-step delay learned system models without effort. Experiments Gym MuJoCo platfo...