Regular points and Green functions in Markov processes

Regular Points of Lipschttz Functions

Let/be a locally Lipschitz function on a Banach space X, and S a subset of X. We define regular (i.e. noncritical) points for / relative to S, and give a sufficient condition for a point z e S to be regular. This condition is then expressed in the particular case when/is C1, and is used to obtain a new proof of Hoffman's inequality in linear programming.

The KTH Visit in Semi-Markov Processes

The value functions of Markov decision processes

We provide a full characterization of the set of value functions of Markov decision processes.

Markov Decision Processes with General Discount Functions

In Markov Decision Processes, the discount function determines how much the reward for each point in time adds to the value of the process, and thus deeply a ects the optimal policy. Two cases of discount functions are well known and analyzed. The rst is no discounting at all, which correspond to the totaland average-reward criteria. The second case is a constant discount rate, which leads to a...

Convergence of Markov Processes Near Saddle Fixed Points

We consider sequences (XN t )t≥0 of Markov processes in two dimensions whose fluid limit is a stable solution of an ordinary differential equation of the form ẋt = b(xt), where b(x) = (−μ 0 0 λ ) x+τ(x) for some λ, μ > 0 and τ(x) = O(|x|2). Here the processes are indexed so that the variance of the fluctuations of XN t is inversely proportional to N . The simplest example arises from the OK Cor...