We consider an approximation scheme for solving Markov Decision Processes (MDPs) with countable state space, finite action space, and bounded rewards that uses an approximate solution of a fixed finite-horizon sub-MDP of a given infinite-horizon MDP to create a stationary policy, which we call “approximate receding horizon control”. We first analyze the performance of the approximate receding h...

An important aspect of numerically approximating the solution of an infinite-horizon optimal control problem is the manner in which the horizon is treated. Generally, an infinite-horizon optimal control problem is approximated with a finite-horizon problem. In such cases, regardless of the finite duration of the approximation, the final time lies an infinite duration from the actual horizon at ...

We address a multi-item capacitated lot-sizing problem with setup times that arises in real-world production planning contexts. Demand cannot be backlogged, but can be totally or partially lost. Safety stock is an objective to reach rather than an industrial constraint to respect. The problem is NP-hard. We propose mixed integer programming heuristics based on a planning horizon decomposition s...

We investigate the computability of problems in probabilistic planning and partially observable innnite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic nite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exist...

In this paper, we study deterministic dynamic lot-sizing problems with service level constraints on the total number of periods in which backorders can occur over the finite planning horizon. We give a natural mixed integer programming formulation for the single item problem (LS-SL-I) and study the structure of its solution. We show that an optimal solution to this problem can be found in O(n2κ...

This study deals with the problem of dependence between production and failure rates within the context of multi part-type production system. It gives an answer about how to produce, i.e. the production rates, and what to produce, i.e. the part-type selection, over a finite horizon of H periods of equal length. We consider a single randomly failing and repairable machine that produces a multi p...

This article investigates the application of optimal feedback control to trajectory planning in voluntary human arm movements. A nonlinear model predictive controller (NMPC) with a finite prediction horizon was used as the optimal feedback controller to predict the hand trajectory planning and execution of planar reaching tasks. The NMPC is completely predictive, and motion tracking or electrom...

Consider a manufacturer serving a set of retail stores each of which faces deterministic demands in a finite planning horizon. At the beginning of the planning horizon, the production capacity of the manufacturer is built, followed by production, outsourcing to third party manufacturers if necessary and distribution to the retail stores. Because the retail stores are usually managed by differen...

The paper deals with an optimal stopping problem with a finite planning horizon where an available search budget, the total amount of money that can be invested in search activities throughout the planning horizon, is limited and where both the probability of an offer being obtained at each point in time and the probability distribution function of an obtained offer's value may depend on the se...

In this paper, we aim to take a step toward a tighter integration of automated planning and Bayesian Optimization (BO). BO is an approach for optimizing costlyto-evaluate functions by selecting a limited number of experiments that each evaluate the function at a specified input. Typical BO formulations assume that experiments are selected one at a time, or in fixed batches, and that experiments...