There is much interest in the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) as a natural Bayesian nonparametric extension of the ubiquitous Hidden Markov Model for learning from sequential and time-series data. However, in many settings the HDP-HMM’s strict Markovian constraints are undesirable, particularly if we wish to learn or encode non-geometric state durations. We can exte...

The notion of infinite order Markov process is introduced and the Markov property of the flow of information is established.

In order to accurately predict the Web service Qos in a highly dynamic environment,we put forward a Qos dynamic prediction method based on Semi-Markov Processes(SMP) and Case-based Reasoning(CBR).This method firstly uses semi-Markov process to predict business state of web service in the future, and then applies the technology of CBR to predict Web service QoS, for example,when the service deal...

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

We introduce the generalized semi-Markov decision process (GSMDP) as an extension of continuous-time MDPs and semi-Markov decision processes (SMDPs) for modeling stochastic decision processes with asynchronous events and actions. Using phase-type distributions and uniformization, we show how an arbitrary GSMDP can be approximated by a discrete-time MDP, which can then be solved using existing M...

In this paper, we present a model and an algorithm for the calculation of the optimal control limit, average cost, sample size, and the sampling interval for an optimal Bayesian chart to control the proportion of defective items produced using a semi-Markov decision process approach. Traditional p-chart has been widely used for controlling the proportion of defectives in various kinds of produc...

This paper introduces a new model for node behavior namely Correlated Node Behavior Model which is an extension of Node Behavior Model. The model adopts semi Markov process in continuous time which clusters the node that has correlation. The key parameter of the process is determined by five probabilistic parameters based on the Markovian model. Computed from the transition probabilities of the...

In this paper we analyze a discrete time single server queue where the service time equals one slot The numbers of arrivals in each slot are assumed to be independent and identically distributed random variables The service process is interrupted by a semi Markov process namely in certain states the server is available for service while the server is not available in other states We analyze bot...