We study long run average behavior of generalized semi-Markov processes with both fixed-delay events as well as variable-delay events. We show that allowing two fixed-delay events and one variable-delay event may cause an unstable behavior of a GSMP. In particular, we show that a frequency of a given state may not be defined for almost all runs (or more generally, an invariant measure may not e...

Abstract For a quasi-regular (symmetric) Dirichlet space (E ,F) and an associated symmetric standard process (Xt, Px), we show that, for u ∈ F , the additive functional u∗(Xt) − u∗(X0) is a semimartingale if and only if there exists an E-nest {Fn} and positive constants Cn such that |E(u, v)| ≤ Cn‖v‖∞, v ∈ FFn,b. In particular, a signed measure resulting from the inequality will be automaticall...

This paper establishes the existence of a solution to the optimality equations in undiscounted semi-Markov decision models with countable state space, under conditions generalizing the hitherto obtained results. In particular, we merely require the existence of a finite set of states in which every pair of states can reach each other via some stationary policy, instead of the traditional and re...

The author’s monograph “Semi-Markov Processes: Application in System Reliability and Maintenance” which will be published by Elsevier in 2014 is presented. The paper is composed of an introduction, the monograph contents, conclusions and the references the monograph contents is based on.

MOTIVATION A basic problem of broad public and scientific interest is to use the DNA of an individual to infer the genomic ancestries of the parents. In particular, we are often interested in the fraction of each parent's genome that comes from specific ancestries (e.g. European, African, Native American, etc). This has many applications ranging from understanding the inheritance of ancestry-re...

As an extension of the discrete-time case, this note investigates the variance of the total cumulative reward for the embedded Markov chain of semi-Markov processes. Under the assumption that the chain is aperiodic and contains a single class of recurrent states recursive formulae for the variance are obtained which show that the variance growth rate is asymptotically linear in time. Expression...

Continuous-time state transition models may end up having large unwieldy structures when trying to represent all relevant stages of clinical disease processes by means of a standard Markov model. In such situations, a more parsimonious, and therefore easier-to-grasp, model of a patient's disease progression can often be obtained by assuming that the future state transitions do not depend only o...