Continuous-time Markov decision processes (CTMDP) may be viewed as a specialcase of semi-Markov decision processes (SMDP) where the intertransition times are exponen-tially distributed and the decision maker is allowed to choose actions whenever the systemstate changes. When the transition rates are identical for each state and action pair, one canconvert a CTMDP into an equival...

Animal biologists commonly use continuous time Markov chain models to describe patterns of animal behaviour. In this paper we consider the use of these models for describing AFL football. In particular we test the assumptions for continuous time Markov chain models (CTMCs), with time, distance and speed values associated with each transition. Using a simple event categorisation it is found that...

Aldous’ spectral gap conjecture asserts that on any graph the random walk process and the random transposition (or interchange) process have the same spectral gap. We prove the conjecture using a recursive strategy. The approach is a natural extension of the method already used to prove the validity of the conjecture on trees. The novelty is an idea based on electric network reduction which red...

Growing amount of information on biological sequences has made application of statistical approaches necessary for modeling and estimation of their functions. In this paper, sensitivity and specificity of the first and second Markov chains for prediction of genes was evaluated using the complete double stranded  DNA virus. There were two approaches for prediction of each Markov Model parameter,...

this paper proposes an extension of markov chain model to characterize of geological formations and to make conditioning on any number of well data possible. in this study, lithofacies cycles of the amiran formation at type and pirshamsedin sections are examined by markov method and entropy analysis as well as field observations, and depositional environment has interpreted by proposed ideal se...

this paper is devoted to the study of determining optimal process mean in system production with the two markets for the sale of goods. in this paper, we developed an absorbing markov chain model in production systems where all items are inspected %100 for conformance with their specification limits. when the value of the quality characteristic of an item falls below a lower limit, the item is ...

Meteorological stations usually contain some missing data for different reasons.There are several traditional methods for completing data, among them bivariate and multivariate linear and non-linear correlation analysis, double mass curve, ratio and difference methods, moving average and probability density functions are commonly used. In this paper a blended model comprising the bivariate expo...