Control of Singularly Perturbed Markov Chains: a Numerical Study

This work is devoted to numerical studies of nearly optimal controls of systems driven by singularly perturbed Markov chains. Our approach is based on the ideas of hierarchical controls applicable to many large-scale systems. A discrete-time linear quadratic control problem is examined. Its corresponding limit system is derived. The associated asymptotic properties and near optimality are demonstrated by numerical examples. Numerical experiments for a continuous-time hybrid linear quadratic regulator with Gaussian disturbances and a discrete-time Markov decision process are also presented. The numerical results have not only supported our theoretical findings but also provided insights for further applications.