We consider a generalized switch model, which includes as special cases the model of multiuser data scheduling over a wireless medium [5, 1], the input-queued cross-bar switch model [3], and a discrete time version of a parallel server queueing system [2, 6]. Input flows, , are served in discrete time by a switch. Switch state follows a finite discrete time Markov chain. In each state , the swi...

The n-coalescent is a continuous-time Markov chain on a finite set of states, which describes the family relationships among a sample of n members drawn from a large haploid population. Its transition probabilities can be calculated from a factorization of the chain into two independent components, a pure death process and a discrete-time jump chain. For a deeper study, it is useful to construc...

We study joint preventive maintenance (PM) and production policies for an unreliable production-inventory system in which maintenance/repair times are non-negligible and stochastic. A joint policy decides (a) whether or not to perform PM, and (b) if PM is not performed, then how much to produce. We consider a discrete-time system, formulating the problem as a Markov decision process (MDP) model...

Continuous-time Markov decision processes (CTMDP) may be viewed as a specialcase of semi-Markov decision processes (SMDP) where the intertransition times are exponen-tially distributed and the decision maker is allowed to choose actions whenever the systemstate changes. When the transition rates are identical for each state and action pair, one canconvert a CTMDP into an equival...

A class of discrete-time priority queueing systems with Markov modulated arrivals is considered. In these systems, N queues are served by a single server according to priorities that are preassigned to the queues. Packet arrivals are modeled as discrete-time batch processes with a distribution that depends on the state of an independent common two-state Markov chain. This allows to cover a wide...

We study the problem of model checking Interval-valued Discrete-time Markov Chains (IDTMC). IDTMCs are discrete-time finite Markov Chains for which the exact transition probabilities are not known. Instead in IDTMCs, each transition is associated with an interval in which the actual transition probability must lie. We consider two semantic interpretations for the uncertainty in the transition p...

Based on the technological process of obtaining tea products, factors determining its quality are considered. The analysis these and their classification. optimization labor costs time to perform control raw materials using methods probability theory. a simple discrete Markov chain, proposed system selective products by variable samples.

For a Markov renewal process where the time parameter is discrete, we present a novel method for calculating the asymptotic variance. Our approach is based on the key renewal theorem and is applicable even when the state space of the Markov chain is countably infinite.

Average cost Markov decision chains with discrete time parameter are considered. The cost function is unbounded and satisfies an additional condition which frequently holds in applications. Also, we assume that there exists a single stationary policy for which the corresponding Markov chain is irreducible and ergodic with finite average cost. Within this framework, the existence of an average c...

We consider the problem of performing inference with imprecise continuous-time hidden Markov chains, that is, imprecise continuous-time Markov chains that are augmented with random output variables whose distribution depends on the hidden state of the chain. The prefix ‘imprecise’ refers to the fact that we do not consider a classical continuous-time Markov chain, but replace it with a robust e...