We study the problem of online learning Markov Decision Processes (MDPs) when both the transition distributions and loss functions are chosen by an adversary. We present an algorithm that, under a mixing assumption, achieves O( √ T log |Π| + log |Π|) regret with respect to a comparison set of policies Π. The regret is independent of the size of the state and action spaces. When expectations ove...

In the paper the integration of available results on SemiMarkov Decision Processes and on Markov Regenerative Processes is attempted, in order to de ne the mathematical framework for solving decision problems where the underlying structure state process is a Markov Regenerative Process, referred to as Markov Regenerative Decision Process. The essential question investigated here is which descri...

We consider the problem of optimally maintaining a periodically inspected system that deteriorates according to a discrete-time Markov process and has a limit on the number of repairs that can be performed before it must be replaced. After each inspection, a decision maker must decide whether to repair the system, replace it with a new one, or leave it operating until the next inspection, where...