We propose empirical dynamic programming algorithms for Markov decision processes (MDPs). In these algorithms, the exact expectation in the Bellman operator in classical value iteration is replaced by an empirical estimate to get ‘empirical value iteration’ (EVI). Policy evaluation and policy improvement in classical policy iteration are also replaced by simulation to get ‘empirical policy iter...

We give multi-stage stochastic programming formulations for lot-sizing problems where costs, demands and order lead times follow a general discrete-time stochastic process with finite support. We characterize the properties of an optimal solution and give a dynamic programming algorithm, polynomial in input size, when orders do not cross in time.

Using the linear programming approach to stochastic control introduced in [6] and [10], we provide a semigroup property for some set of probability measures leading to dynamic programming principles for stochastic control problems. An abstract principle is provided for general bounded costs. Linearized versions are obtained under further (semi)continuity assumptions.

a linear programming model has been presented for finding an appropriate planning in order to maximize hotel revenue. in this model, a special planning horizon has been considered that includes several busy days of a year. there are several reservation periods that may start a few months earlier. each period of reservation may have different prices and so result different incomes. hotel custome...

In this paper we discuss risk neutral and risk averse approaches to multistage (linear) stochastic programming problems based on the Stochastic Dual Dynamic Programming (SDDP) method. We give a general description of the algorithm and present computational studies related to planning of the Brazilian interconnected power system. 2012 Elsevier B.V. All rights reserved.

In this paper, a single-product, single-machine system under Markovian deterioration of machine condition and demand uncertainty is studied.  The objective is to find the optimal intervals for inspection and preventive maintenance activities in a condition-based maintenance planning with discrete monitoring framework. At first, a stochastic dynamic programming model whose state variable is the ...

In this paper we discuss computational complexity and risk averse approaches to two and multistage stochastic programming problems. We argue that two stage (say linear) stochastic programming problems can be solved with a reasonable accuracy by Monte Carlo sampling techniques while there are indications that complexity of multistage programs grows fast with increase of the number of stages. We ...