In this work we present a methodology for estimating the variability of a signal modeled as a continuous time stochastic process observable only at discrete times with variability modeled by a continuous time Markov chain. The methodology estimates the parameters of the hidden Markov chain. The methodology is new however the major contribution of this work comes in the realm of applications. Th...

In this paper the equivalence between various divisible load-scheduling policies and continuous time Markov chains is demonstrated. The problem is to show optimal divisible load schedules for various network topologies have Markov chain analogs. This paper is a continuation of our initial short paper [1] that introduced this unification between divisible load theory and Markov chain models for ...

This deliverable describes the first year results of the QUASIMODO project on analysing quantitative systems. Keyword list: Markov chain, Markov decision process, probabilistic bisimulation, probabilistic simulations; probabilistic timed automata; priced probabilistic timed automata; Continuous Time Markov Chains, inhomogeneous CTMC, Infinite state CTMC, counter-example guided abstractionrefine...

This paper studies the optimum stochastic control problem with piecewise deterministic dynamics. The controls enter through the system dynamics as well as the transitions for the underlying Markov chain process, and are allowed to depend on both the continuous state and the current state of the Markov chain. The paper shows that the feedback optimal control relies on the viscosity solutions of ...

in this paper, the ambiguity of nite state irreducible markov chain trajectories is reminded and is obtained for two state markov chain. i give an applicable example of this concept in president election

with recent developments in communication technologies, it is possible to access information any time and in any place; however, mobile environments bring forward certain requirements and exclusive challenges including low connection speeds and bandwidth limitations. thus, we need to consider mobile limitations when using web services. providing services and limited resource allocation to users...

Abstract Two GI/M/1 type Markov chains associated with the queue length are often used in analyzing the discrete time MAP/PH/K queue. The first Markov chain is introduced by tracking service phases for servers; a method we call TPFS. The transition probability matrix of the Markov chain can be constructed in a straightforward manner. The second Markov chain is introduced by counting servers for...