High-level semi-Markov modelling paradigms (such as semi-Markov stochastic Petri nets and process algebras) are used to capture realistic performance models of computer and communication systems but have the drawback of generating huge underlying semi-Markov state spaces. Extraction of measures such as steady-state probabilities and passage-time distributions relies on sparse matrix representat...

This paper present methodological results that allow the cost-effective numerical analysis of finite-state generalized semi-Markov processes (GSMPs) with exponential and deterministic events by an embedded general state space Markov chain (GSSMC). Key contributions constitute the formal proof that elements of the transition kernel of the GSSMC can always be computed by appropriate summation of ...

We extend the definition of level-crossing ordering of stochastic processes, proposed by Irle andGani (2001), to the case in which the times to exceed levels are compared using an arbitrary stochastic order, and work, in particular, with integral stochastic orders closed for convolution. Using a sample-path approach, we establish level-crossing ordering results for the case in which the slower ...

We develop the theory for Markov and semi-Markov control using dynamic programming and reinforcement learning in which a form of semi-variance which computes the variability of rewards below a pre-specified target is penalized. The objective is to optimize a function of the rewards and risk where risk is penalized. Penalizing variance, which is popular in the literature, has some drawbacks that...

^ The Semi-Markov Decision model is considered under the criterion of long-run average cost. A new criterion, which for any policy considers the limit of the expected cost Incurred during the first n transitions divided by the expected length of the first n transitions, is considered. Conditions guaranteeing that an optimal stationary (nonrandomized) policy exist are then presented. It is also ...

We introduce the minimal maximally predictive models ( -machines) of processes generated by certain hidden semi-Markov models. Their causal states are either hybrid discrete-continuous or continuous random variables and causal-state transitions are described by partial differential equations. Closed-form expressions are given for statistical complexities, excess entropies, and differential info...