Karl Sigman 1 Gambler ’ s Ruin Problem

Let N ≥ 2 be an integer and let 1 ≤ i ≤ N − 1. Consider a gambler who starts with an initial fortune of $i and then on each successive gamble either wins $1 or loses $1 independent of the past with probabilities p and q = 1 − p respectively. Let Xn denote the total fortune after the nth gamble. The gambler’s objective is to reach a total fortune of $N , without first getting ruined (running out of money). If the gambler succeeds, then the gambler is said to win the game. In any case, the gambler stops playing after winning or getting ruined, whichever happens first. {Xn} yields a Markov chain (MC) on the state space S = {0, 1, . . . , N}. The transition probabilities are given by Pi,i+1 = p, Pi,i−1 = q, 0 < i < N , and both 0 and N are absorbing states, P00 = PNN = 1.1 For example, when N = 4 the transition matrix is given by