We consider the problem of performing inference with imprecise continuous-time hidden Markov chains, that is, imprecise continuous-time Markov chains that are augmented with random output variables whose distribution depends on the hidden state of the chain. The prefix ‘imprecise’ refers to the fact that we do not consider a classical continuous-time Markov chain, but replace it with a robust e...

Stochastic process algebras (e.g. PEPA [10], EMPA [1], TIPP [9]) emerged about 15 years ago as system description techniques for performance modelling. They have enjoyed some considerable success in this arena. For example, PEPA has been used to study the performance of a wide variety of systems [12, 2, 3, 18, 13]. This analysis has been based on the generation of a continuous time Markov chain...

We consider a Hidden Markov Model (HMM) where the integrated continuous-time Markov chain can be observed at discrete time points perturbed by a Brownian motion. The aim is to derive a filter for the underlying continuous-time Markov chain. The recursion formula for the discrete-time filter is easy to derive, however involves densities which are very hard to obtain. In this paper we derive exac...

Motivated by queues with many-servers, we study Brownian steady-state approximations for continuous time Markov chains (CTMCs). Our approximations are based on diffusion models (rather than a diffusion limit) whose steady-state, we prove, approximates well that of the Markov chain. Strong approximations provide such “limitless” approximations for process dynamics. Our focus here is on steady-st...

1 Basic aspects of continuous time Markov chains 3 1.1 Markov property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 Regular jump chain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 1.3 Holding times . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.4 Poisson process . . . . . . . . . . . . . . ....

In this paper, we develop approximation methods for solving stochastic control problems. The systems under consideration involve regime switching, modulated by a continuous-time Markov chain. Using Markov chain approximation techniques, we construct discrete-time Markov chains having two components. In addition to convergence of the procedure, numerical experiments and remarks on controlled var...

Modelling of a mobile ad-hoc network is a simplest way to represent real life networks for reliability and performance analysis. In this paper, an effort is made to develop a model of ad-hoc network cluster for evaluation of reliability. This model is based on Continuous-Time Markov Chain (CTMC). A mobile ad-hoc network model is especially useful to understand the practical implications and lim...

The usage control (UCON) model demands for continuous control over objects of a system. Access decisions are done several times within a usage session and are performed on the basis of mutable attributes. Values of attributes in modern highly-dynamic and distributed systems sometimes are not up-to-date, because attributes may be updated by several entities and reside outside the system domain. ...