In this paper we consider a single buffer fluid model where sources arrive according to a Poisson process, and stay in the system for independent and identically distributed (iid) exponential amounts of time and then depart. While they are in the system, they generate fluid traffic that is modulated by iid finite state continuous time Markov chains (CTMC). The fluid in the buffer is drained at ...

In this paper, we investigate the exact controllability properties of an advection-diffusion equation on a bounded domain, using timeand space-dependent velocity fields as the control parameters. This partial differential equation (PDE) is the Kolmogorov forward equation for a reflected diffusion process that models the spatiotemporal evolution of a swarm of agents. We prove that if a target pr...

Computer systems are ubiquitous in almost all spheres of our life, motivating the need for them to function correctly and in a timely manner. Continuous time Markov chains (CTMCs) are a widely used formalism for the performance analysis of computer systems. A large variety of useful performance measures can be derived from a CTMC via the computation of its steady-state probabilities. Traditiona...

This paper, the second of a two-part series, presents a method for mean-field feedback stabilization of a swarm of agents on a finite state space whose time evolution is modeled as a continuous time Markov chain (CTMC). The resulting (mean-field) control problem is that of controlling a nonlinear system with desired global stability properties. We first prove that any probability distribution w...

a r t i c l e i n f o a b s t r a c t MSC: 34D20 34K50 60H10 93E15 Keywords: Pure delay system Two-timescale Markov chain Stationary distribution Almost sure uniform stability Yorke's condition This work examines almost sure stability of a pure random delay system whose delay time is modeled by a finite state continuous-time Markov chain with two-time scales. The Markov chain contains a fast-va...

The aim of this paper is to model and evaluate the performance of intrusion detection systems (IDSs) facing black-hole and gray-hole attacks within wireless ad hoc networks (WANETs). The main performance metric of an IDS in a WANET can be defined as the mean time required for the IDS to detect an attack in the network. To evaluate this measure, two types of stochastic models called continuous t...

The aim of this thesis is to investigate the mathematics of Markov-modulated models for derivatives pricing. We consider a model where instantaneous stock volatility and drift are driven by a continuous time finite Markov chain. We present a new derivation of an integral representation for attainable non-path dependent options’ prices in a twostate and three-state Markov chain model, and comput...

The covariance ordering, for discrete and continuous time Markov chains, is defined and studied. This partial ordering gives a necessary and sufficient condition for MCMC estimators to have small asymptotic variance. Connections between this ordering, eigenvalues, and suprema of the spectrum of the Markov transition kernel, are provided. A representation of the asymptotic variance of MCMC estim...

Continuous-time Markov chains are used extensively to analyze the performance of various computer networks. However, constructing and solving continuous-time Markov chain is a tedious and error-prone procedure, especially when the studied systems are complex. Stochastic Petri nets and the corresponding software packages provide automated generation and solution to continuous-time Markov chains....