Limiting Covariance in Markov-renewal Processes

General additive functions called rewards are defined on a "regular" finite-state Markov-renewal process. The asymptotic form of the mean total reward in [0,t] has previously been obtained, and it is known that the total rewards are joint-normally distributed as t -► oo. This paper finds the dominant asymptotic term in the covariance of the total rewetrds as a simple function of the moments of the per-transition rewards, and the "bias" term of the mean total rewards. Special formulas for the dominant covariance term of "number of visits", and "occupation time" in given states are also derived. LIMITING COVARIANCE IN M/JKOV RENEWAL PROCESSES Consider a finite-state Markov-renewal process which moves through states V^'-'-VSc+l'--at tlnie8 o-<i<--k<k+i<"If a reward is earned during each transition from state to state, and if successive rewards are additive, it is of interest to study the total reward earned during the interval [0,t]. Typically, the reward earned during a transition from i, to i, might be a random variable which depends upon the values of i, , i, , , S. ,S. , as well as on the "excess time" t-S (S. ^ t < S.+1) of any uncompleted transition. Thus, the total reward earned in [0,t] is a random sum of additive random variables, and has a well-defined, though complicated distribution. The purpose of this paper is to summarize some known results on the asymptotic form of the mean total reward, and to present some new results on the dominant asymptotic term of the (co-)variance of the total reward. These results are useful primarily because a central limit theorem often holds for the distribution of total reward, as t -♦ w . DEFINITIONS, NOTATION, AND SUMMARY OF RESULTS The definition of a Markov-renewal process is that: ^k+lJ i \+l ^ X + \ I ^ • i • W-! ' ^ ' V \.l'--l' cf k=0,l,2,.. r K U I Z 1 = QijCx) = Pi/ijCx) |i,J=l,2,...M j In other words, the process may be considered as an imbedded Markov chain in which the movement between the M states is governed by the