In this paper, the controller synthesis problem for fault tolerant control systems (FTCS) with stochastic stability and H2 performance is studied. The system faults of random nature are modeled by a Markov chain. Because the real system fault modes are not directly accessible in the context of FTCS, the controller is reconfigured based on the output of a Fault Detection and Identification (FDI)...

An overview of statistical and information-theoretic aspects of hidden Markov processes (HMPs) is presented. An HMP is a discrete-time finite-state homogeneous Markov chain observed through a discrete-time memoryless invariant channel. In recent years, the work of Baum and Petrie on finite-state finite-alphabet HMPs was expanded to HMPs with finite as well as continuous state spaces and a gener...

Suppose we observe a discrete-time Markov chain at certain periodic or random time points only. Which observation patterns allow us to identify the transition distribution? In case we can identify it, how can we construct (good) estimators? We discuss these questions both for nonparametric models and for linear autoregression.

A novel approach for obtaining the response time in a discrete-time tandem-queue with blocking is presented. The approach constructs a Markov chain based on the age of the leading customer in the first queue. We also provide a stability condition and carry out several numerical examples. © 2004 Elsevier B.V. All rights reserved.

[Tip: Study the MC, QT, and Little's law lectures together: CTMC (MC lecture), M/M/1 queue (QT lecture), Little's law lecture (when deriving the mean response time from mean number of customers), DTMC (MC lecture), M/M/1 queue derivation using DTMC analysis, derive distribution of response time in M/M/1 queue (QT lecture), relation between Markov property and mem-oryless property (MC lecture), ...

Digraph panel data, corresponding to a given set of nodes and the directed graphs (digraphs) on the set of nodes which are observed at two or more discrete time points, are collected in the social sciences and other fields. Conventional models of digraph panel data assume that the data are discrete outcomes of a continuous-time Markov process on the set of possible digraphs defined on the set o...

We consider discrete-time Markov chains with one coffin state and a finite set S of transient states, and are interested in the limiting behaviour of such a chain as time n tends to infinity, conditional on survival up to n. It is known that, when S is irreducible, the limiting conditional distribution of the chain equals the (unique) quasi-stationary distribution of the chain, while the latter...

We propose a random adaptation variant of time-varying distributed averaging dynamics in discrete time. show that this leads to novel interpretations fundamental concepts averaging, opinion dynamics, and learning. Namely, we the ergodicity stochastic chain is equivalent almost sure ( xmlns:xlin...

This paper studies the effect of bisimulation minimisation in model checking of monolithic discrete-time and continuous-time Markov chains as well as variants thereof with rewards. Our results show that—as for traditional model checking—enormous state space reductions (up to logarithmic savings) may be obtained. In contrast to traditional model checking, in many cases, the verification time of ...