The cover time of a Markov chain on finite state space is the expected until all states are visited. We show that if discrete-time with rational transitions probabilities bounded, then it number. result proved by relating original to hitting set in another higher dimensional chain. prove this more general setting where k≥1 independent copies run simultaneously same space.

Today many formalisms exist for specifying complex Markov chains. In contrast, formalism for specifying the quantitative properties to analyze have remained quite primitive. In this paper a new formalism of temporal rewards that allows complex quantitative properties (including delay type measures) to be expressed in the form of a temporal reward formula. Together, an initial (discrete-time) Ma...

In this paper we describe a two-step scheme for approximate model checking of discrete time stochastic hybrid systems. In the first step, the stochastic hybrid system is approximated by a finite state Markov chain. In the second step, the Markov chain is model checked for the desired property. In particular, we consider the probabilistic invariance property and show that, under certain regulari...

After successfully completing this module an APTS student will be able to: I describe and calculate with the notion of a reversible Markov chain, both in discrete and continuous time; I describe the basic properties of discrete-parameter martingales and check whether the martingale property holds; I recall and apply some signi cant concepts from martingale theory; I explain how to use Foster-Ly...

Hybrid systems whose mode dynamics are governed by nonlinear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a probabilistic analysis method by approximating the mode transitions as stochastic events. We assume that the probability of making a mode transition is proportional to t...

Hidden Markov Model is a popular statisical method that is used in continious and discrete speech recognition. The probability density function of observation vectors in each state is estimated with discrete density or continious density modeling. The performance (in correct word recognition rate) of continious density is higher than discrete density HMM, but its computation complexity is very ...

In this note, some testable conditions for mean square (i.e., second moment) stability for discrete-time jump linear systems with timehomogenous and time-inhomogenous finite state Markov chain form processes are presented.