The majority of computational methods applied for the analysis of homogeneous Markov reward models (MRMs) are not applicable for the analysis of inhomogeneous MRMs. By the nature of inhomogeneous models only forward differential equations can be used to describe the model behaviour. In this paper we provide forward partial differential equations describing the distribution of reward measures of...

Stochastic process algebras, such as PEPA, provide a novel approach to performance modelling. As well as facilitating a composi-tional approach, process algebra models focus on a system's behaviour rather than its state space. Classical process algebras are complemented by modal and temporal logics which concisely express possible model behaviours. These logics are widely used during functional...

We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...

We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...

We investigate the computability of problems in probabilistic planning and partially observable infinite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there ex...

In this tutorial, we discuss several practical issues regarding speci cation and solution of dependability and performability models. We compare model types with and without rewards. Continuous-time Markov chains (CTMCs) are compared with (continuous-time) Markov reward models (MRMs) and generalized stochastic Petri nets (GSPNs) are compared with stochastic reward nets (SRNs). It is shown that ...

In this tutorial, we discuss several practical issues regarding specification and solution of dependability and performability models. We compare model types with and without rewards. Continuous-time Markov chains (CTMCs) are compared with (continuous-time) Markov reward models (MRMs) and generalized stochastic Petri nets (GSPNs) are compared with stochastic reward nets (SRNs). It is shown that...

We consider risk minimization problems for Markov decision processes. From a standpoint of making the risk of random reward variable at each time as small as possible, a risk measure is introduced using conditional value-at-risk for random immediate reward variables in Markov decision processes, under whose risk measure criteria the risk-optimal policies are characterized by the optimality equa...