The authors consider the fundamental problem of nding good policies in uncertain models. It is demonstrated that although the general problem of nding the best policy with respect to the worst model is NP-hard, in the special case of a convex uncertainty set the problem is tractable. A stochastic dynamic game is proposed, and the security equilibrium solution of the game is shown to correspond ...

We introduce Markovian cocycle perturbations of the groups of transformations associated with the classical and quantum stochastic processes with stationary increments, which are characterized by a localization of the perturbation to the algebra of events of the past. It is namely the definition one needs because the Markovian perturbations of the Kolmogorov flows associated with the classical ...

We examine technologies designed to solve decision processes with non-Markovian rewards (NMRDPs). More specifically, target decision processes exhibit Markovian dynamics, called grounded dynamics, and desirable behaviours are modelled as state trajectories specified in a temporal logic. Each technology operates by automatically translating NMRDPs into corresponding equivalent MDPs amenable to c...

The Petri Box Calculus (PBC) combines two well known paradigms of the design of concurrent systems: process algebras and Petri nets. In our first proposal of sPBC (stochastic PBC) [12] we defined a Markovian extension of finite PBC, i.e., we had a Markovian process algebra for which both an operational and a denotational (based on stochastic Petri nets) semantics were defined. Our goal in this ...

In this paper a bond market model and the related term structure of interest rates are studied where prices of zero coupon bonds are driven by a jump-diffusion process. A criterion is derived on the deterministic forward rate volatilities under which the short rate process is Markovian. In the case that the volatilities depend on the short rate sufficient conditions are presented for the existe...

In this paper a method is proposed called circulant matching method to approximate the superposition of a number of discrete time batch Markovian arrival sources by a circulant batch Markovian process while matching the stationary cumulative distribution and the autocorrelation sequence of the input rate process Special attention is paid to periodic sources The method is applied to the superpos...

Markovian process calculi constitute a useful framework for reasoning about the functional and performance aspects of concurrent systems. This is achieved by means of behavioral equivalences that take into account both the action names and their exponentially distributed durations. A notable extension to the expressiveness of Markovian process calculi derives from the adoption of GSPNlike immed...

We consider a Markovian regime-switching risk model (also called the Markov-modulated risk model) with stochastic premium income, in which the premium income and the claim occurrence are driven by the Markovian regime-switching process. The purpose of this paper is to study the integral equations satisfied by the expected discounted penalty function. In particular, the discount interest force p...