When Bellman's Principle Fails
نویسنده
چکیده
We present several examples which show that the well known statements about Markov Decision Processes can fail if the loss function is not bounded.
منابع مشابه
On Bellman's principle with inequality constraints
We consider an example by Haviv (1996) of a constrained Markov decision process that, in some sense, violates Bellman’s principle. We resolve this issue by showing how to preserve a form of Bellman’s principle that accounts for a change of constraint at states that are reachable from the initial state.
متن کاملExistence and Exponential Stability of Positive Almost Periodic Solutions for a Model of Hematopoiesis
By employing the contraction mapping principle and applying Gronwall-Bellman's inequality, sufficient conditions are established to prove the existence and exponential stability of positive almost periodic solution for nonlinear impulsive delay model of hematopoiesis.
متن کاملSentence-hypotheses generation in a continuous-speech recognition system
In this paper, the dynamic-programming algorithm for continuous-speech recognition is modified in orderto obtain a top-N sentence-hypotheses Iist instead of the usual one sentence only. The theoretical basis of this extension is a generalization of Bellman's principle of optimality. Due to the computational complexity of the new algorithm, a sub-optimal variant is proposed, and experimental res...
متن کاملImplementing Resolute Choice Under Uncertainty
The adaptation to situations of sequential choice under uncertainty of decision criteria which deviate from (subjective) expected utility raises the problem of ensuring the selection of a non dominated strategy. In particular, when following the suggestion of Machina and McClennen of giving up separability (also known as consequentialism), which requires the choice of a substrategy in a subtre...
متن کاملStochastic optimal control via Bellman's principle
This paper presents a method for finding optimal controls of nonlinear systems subject to random excitations. The method is capable to generate global control solutions when state and control constraints are present. The solution is global in the sense that controls for all initial conditions in a region of the state space are obtained. The approach is based on Bellman’s Principle of optimality...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2009