Steady-State Planning in Expected Reward Multichain MDPs

The planning domain has experienced increased interest in the formal synthesis of decision-making policies. This typically entails finding a policy which satisfies specifications form some well-defined logic. While many such logics have been proposed with varying degrees expressiveness and complexity their capacity to capture desirable agent behavior, value is limited when deriving policies satisfy certain types asymptotic behavior general system models. In particular, we are interested specifying constraints on steady-state an agent, captures proportion time spends each state as it interacts for indefinite period its environment. sometimes called average or expected associated problem faced significant challenges unless strong restrictions imposed underlying model terms connectivity graph structure. this paper, explore that consists satisfied. A linear programming solution case multichain Markov Decision Processes (MDPs) prove optimal solutions programs yield stationary rigorous guarantees behavior.