A Markov chain is a discrete time Markov process, i.e. the future of the process is determined by the present and independent of the past. Definition 1.2. A Markov process is time-homogeneous if there exists a measurable map P from X into P (X ), the space of probability measures on X , such that P(Xn ∈ A|Xn−1 = a) = P (a;A) for every A ∈ B(X ), almost every a ∈ X , and every n > 0. We call P t...

T HE discrete memoryless broadcast channel (%P(YJlX), 94 x 3) consists of three finite sets % ., 9, % and a probability transition matrixp(y,zjx). Let p,(ylx) andp,(zIx) be the two marginals of p(y,zlx), and let P, and P2 denote the discrete memoryless channels with probability transition matrices p&y Ix) and pz(zlx), respectively. Recall the following three relations between P, and P2. Definit...

This paper focuses on stability and H∞ performance analysis for a class of continuous-time Markovian jump systems with unknown transition probability and time-varying delay. Based on the free-connection weighting matrix method, the less conservative stability criterion and bounded real lemma (BRL) of Markovian jump systems with time-varying delay are obtained by considering the influence of par...

We consider the problem of undiscounted reinforcement learning in continuous state space. Regret bounds in this setting usually hold under various assumptions on the structure of the reward and transition function. Under the assumption that the rewards and transition probabilities are Lipschitz, for 1-dimensional state space a regret bound of Õ(T 3 4 ) after any T steps has been given by Ortner...

We apply the Wang-Landau method to the study of the critical behavior of the three dimensional Random Field Ising Model with a bimodal probability distribution. For high values of the random field intensity we find that the energy probability distribution at the transition temperature is double peaked, suggesting that the phase transition is first order. On the other hand, the transition looks ...

SOFT HANDOFF MODELING IN CDMA CELLULAR SYSTEMS Jae Kyun Kwon and Dan Keun Sung Department of Electrical Engineering, Korea Advanced Institute of Science and Technology, Korea Abstract|We propose a new analytical soft hando model in a rectangular cell con guration and obtain the probability distribution of the sojourn time in the soft hando region. The movements of a mobile user are categorized ...