The optimization problem of general utility case is considered for countable state semi-Markov decision processes. The regret-utility function is introduced as a function of two variables, one is a target value and the other is a present value. We consider the expectation of the regret-utility function incured until the reaching time to a given absorbing set. In order to characterize the regret...

Semi-Markov Stochastic Petri Nets (SM-SPNs) are a highlevel formalism for defining semi-Markov processes. We present an extended Continuous Stochastic Logic (eCSL) which provides an expressive way to articulate performance queries at the SM-SPN model level. eCSL supports queries involving steady-state, transient and passage time measures. We demonstrate this by formulating and answering eCSL qu...

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

The paper examines the application of semi-Markov models to the phenomenon of earthquakes in Tehran province. Generally, earthquakes are not independent of each other, and time and place of earthquakes are related to previous earthquakes; moreover, the time between earthquakes affects the pattern of their occurrence; thus, this occurrence can be likened to semi-Markov models. ...

In 2007, Abdulla et al. introduced the elegant concept of decisive Markov chain. Intuitively, decisiveness allows one to lift the good properties of finite Markov chains to infinite Markov chains. For instance, the approximate quantitative reachability problem can be solved for decisive Markov chains (enjoying reasonable effectiveness assumptions) including probabilistic lossy channel systems a...

the notion of infinite order markov process is introduced and the markov property of the flow of information is established.

A generalized semi-Markou scheme models the structure of a discrete event system, such as a network of queues. By studying combinatorial and geometric representations of schemes we find conditions for second-order properties-convexity/concavity, sub/supermodularity-of their event epochs and event counting processes. A scheme generates a language of feasible strings of events. We show that monot...

The interference that results from processors attempting to simultaneously access the same main memory in a multiprocessor can be reduced by constructing the memory from separate modules accessible through a crossbar network. The effectiveness of this solution depends on the number of processors and the number of memory modules, and on the parameters of the computation being executed, such as t...

The analysis of correlated point process data has wide applications, ranging from biomedical research to network analysis. In this work, we model such data as generated by a latent collection of continuous-time binary semi-Markov processes, corresponding to external events appearing and disappearing. A continuous-time modeling framework is more appropriate for multichannel point process data th...