These lecture notes present four Markovian models: discrete time Markov chains (DTMC), continous time Markov chains (CTMC), Markov decision processes (MDP) and probabilistic automata (PA). It is addressed for master students and tries as most as possible to be self-contained. However the basics of discrete probability (and additionally the basics of measure and integration for the study of CTMC...

Markovian Process Algebra (MPA) is a process algebra enhanced with exponential timing which allows the mapping of speciications on continuous time Markov chains (CTMCs). This paper introduces a compositional approach to compute the generator matrix of the CTMC underlying a MPA speciication which consists of the parallel composition of nite state agents. Furthermore two diierent equivalence rela...

This paper presents an Interval Decision Diagram (IDD) based approach to realize symbolically transient analysis of Continuous Time Markov Chains (CTMC) which are derived from stochastic Petri nets. Matrix-vector and vector-matrix multiplication are the major tasks when doing exact analysis of CTMCs. We sketch a simple algorithm which uses explicitly the Petri net structure and offers the oppor...

This paper proposes a novel abstraction technique for continuous-time Markov chains (CTMCs). Our technique fits within the realm of three-valued abstraction methods that have been used successfully for traditional model checking. The key idea is to apply abstraction on uniform CTMCs that are readily obtained from general CTMCs, and to abstract transition probabilities by intervals. It is shown ...

We illustrate a framework for simulating degradation processes in components of energy systems. The logical framework revolves around a multi-state representation of the processes. The physics governing the processes is captured in the transition rates governing the random walk across the degradation states. The computational framework relies on a representation of the process dynamics by stoch...

Uniformisation[1, 2] is a key technique which allows modellers to extract passage-time quantiles/densities which in turn permits the plotting of probability density and cumulative distribution functions. Uniformisation converts a CTMC (Continuous-Time Markov Chain) into a DTMC (Discrete-Time Markov Chain) with equivalent semantics. This can be used to calculate the probability of completing a p...