We consider a semi-Markov modulated security market consisting of a riskless asset or bond with constant interest rate and risky asset or stock, whose dynamics follow gemoetric Brownian motion with volatility that depends on semi-Markov process. Two cases for semi-Markov volatilities are studied: local current and local semi-Markov volatilities. Using the martingale characterization of semi-Mar...

Multivariate reward processes with reward functions of constant rates, defined on a semi-Markov process, first were studied by Masuda and Sumita, 1991. Reward processes with nonlinear reward functions were introduced in Soltani, 1996. In this work we study a multivariate process , , where are reward processes with nonlinear reward functions respectively. The Laplace transform of the covar...

We investigate Semi-Markov Decision Processes (SMDPs). Two problems are studied, namely, the time-bounded reachability problem and the long-run average fraction of time problem. The former aims to compute the maximal (or minimum) probability to reach a certain set of states within a given time bound. We obtain a Bellman equation to characterize the maximal time-bounded reachability probability,...

A semi-Markov process stays in state x for a time s and then jumps to state y according to a transition distribution Q(x; dy; ds). A statistical model is described by a family of such transition distributions. We give conditions for a nonparametric version of local asymptotic normality as the observation time tends to innnity. Then we introducèempirical' estimators for linear functionals of the...

We consider semiparametric models of semi-Markov processes with arbitrary state space. Assuming that the process is geometrically ergodic, we characterize efficient estimators, in the sense of Hájek and Le Cam, for arbitrary real-valued smooth functionals of the distribution of the embedded Markov renewal process. We construct efficient estimators of the parameter and of linear functionals of t...

First-passage time densities and quantiles are important metrics in performance analysis. They are used in the analysis of mobile communication systems, web servers, manufacturing systems as well as for the analysis of the quality of service of hospitals and government organisations. In this report we look at computational techniques for the first-passage time analysis on highlevel models that ...

In multiple criteria Markov Decision Processes (MDP) where multiple costs are incurred at every decision point, current methods solve them by minimising the expected primary cost criterion while constraining the expectations of other cost criteria to some critical values. However, systems are often faced with hard constraints where the cost criteria should never exceed some critical values at a...

With the increasing complexity of multiprocessor and distributed processing systems, the need to develop efficient and accurate modeling methods is evident. Fault tolerance and degradable performance of such systems has given rise to considerable interest in models for the combined evaluation of performance and reliability [l], [2]. Markov or semi-Markov reward models can be used to evaluate th...

We develop an Onsager-Machlup-type theory for nonequilibrium semi-Markov processes. Our main result is an exact large time asymptotics for the joint probability of the occupation times and the currents in the system, establishing some generic large deviation structures. We discuss in detail how the nonequilibrium driving and the non-exponential waiting time distribution influence the occupation...