On Partially Observed Stochastic Shortest Path Problems

We analyze a class of partially observed stochastic shortest path problems. These are terminating Markov decision process with imperfect state information that evolve on an innnite time horizon and have a total cost criterion. For well-posedness, we make reasonable stochastic shortest path type assumptions: (1) the existence of a policy that guarantees termination with probability one and (2) the property that any policy that fails to guarantee termination has innnite expected cost from some initial state. We also assume that termination is perfectly recognized. We establish the existence of a stationary optimal policy along with the existence of a unique bounded solution to Bellman's equation. We also reveal the convergence properties of value and policy iteration. For the case where policies exist that do not guarantee termination, the dynamic programming operator fails to have an m-stage contraction property.