Approximate dynamic programming has evolved, initially independently, within operations research, computer science and the engineering controls community, all searching for practical tools for solving sequential stochastic optimization problems. More so than other communities, operations research continued to develop the theory behind the basic model introduced by Bellman with discrete states a...

Inference is a key component in learning probabilistic models from partially observable data. When learning temporal models, each of the many inference phases requires a traversal over an entire long data sequence; furthermore, the data structures manipulated are exponentially large, making this process computationally expensive. In [2], we describe an approximate inference algorithm for monito...

Stochastic optimization problems pose unique challenges in how they are represented mathematically. These problems arise in a number of different communities, often in the context of problems that introduce specific computational characteristics. As a result, a number of contrasting notational styles have evolved, which complicate our ability to communicate research across communities. This is ...

OPTIMIZATION-BASED APPROXIMATE DYNAMIC PROGRAMMING

This paper develops theoretical foundations for an error analysis of approximate equilibria in dynamic stochastic general equilibrium models with heterogeneous agents and incomplete financial markets. While there are several algorithms which compute prices and allocations for which agents’ first order conditions are approximately satisfied (‘approximate equilibria’), there are few results on ho...

In conventional power plants trash-racks are provided at the intakes to protect the turbines. In pumped storage plants, the draft tube or tailrace must also have trash-racks to protect the units while pumping. Because the loads believed to cause many failures of trash-racks are dynamic in nature, it is important to understand the dynamic characteristics of trash-rack structures in general and a...

We present tractable algorithms to assess the sensitivity of a stochastic dynamic fleet management model to fleet size and load availability. In particular, we show how to compute the change in the objective function value in response to an additional vehicle or an additional load introduced into the system. The novel aspect of our approach is that it does not require multiple simulations with ...

This paper provides an analysis of error propagation in Approximate Dynamic Programming applied to zero-sum two-player Stochastic Games. We provide a novel and unified error propagation analysis in Lp-norm of three well-known algorithms adapted to Stochastic Games (namely Approximate Value Iteration, Approximate Policy Iteration and Approximate Generalized Policy Iteratio,n). We show that we ca...