Recent work by Han and Van Roy (2011) introduced a linear programming technique to compute good sub-optimal solutions to high-dimensional control problems in a diffusion-based setting. Their problem formulation worked with finite horizon problems where the horizon, T , is an exponentially-distributed random variable. We extend their approach to finite horizon problems with a fixed horizon T . W...

This article provides importance sampling algorithms for computing the probabilities of various types ruin of spectrally negative Lévy risk processes, which are ruin over the infinite time horizon, ruin within a finite time horizon and ruin past a finite time horizon. For the special case of the compound Poisson process perturbed by diffusion, algorithms for computing probabilities of ruins by ...

Minimum cost flow problems in infinite networks arise, for example, in infinite-horizon sequential decision problems such as production planning. Strong duality for these problems was recently established for the special case of linear costs using an infinite-dimensional simplex algorithm. Here, we use a different approach to derive duality results when the costs are convex. We formulate the pr...

Based on recent results for multi-armed bandit problems, we propose an adaptive sampling algorithm that approximates the optimal value of a finite horizon Markov decision process (MDP) with infinite state space but finite action space and bounded rewards. The algorithm adaptively chooses which action to sample as the sampling process proceeds, and it is proven that the estimate produced by the ...

We prove a general theorem that the L2ρ(R ;R) ⊗ L2ρ(R ;R) valued solution of an infinite horizon backward doubly stochastic differential equation, if exists, gives the stationary solution of the corresponding stochastic partial differential equation. We prove the existence and uniqueness of the L2ρ(R ;R)⊗Lρ(R ;R) valued solutions for backward doubly stochastic differential equations on finite a...

We develop a general theory of efficient policy gradient algorithms for Noise-Action MDPs (NMDPs), a class of MDPs that generalize Linearly Solvable MDPs (LMDPs). For finite horizon problems, these lead to simple update equations based on multiple rollouts of the system. We show that our policy gradient algorithms are faster than the PI algorithm, a state of the art policy optimization algorith...

This paper presents an efficient algorithin for coiiiputing the solution to the constrained infinite time linear quadratic regulator (CLQR) problem for discrete time systems. The algorithm coinbiiies multi-parametric quadratic programming with reachability analysis to obtain the optiinal piecewise affine (PWA) feedback law. The algorithm reduces the time necessary to compute the PWA solution fo...

In this paper we model production problems where yields are stochastic, demands are substitutable, and several items are jointly produced. We have formulated this problem as a profit maximizing convex program, and have studied two solution procedures. The first method solves finite horizon stochastic programs on a rolling horizon basis. We have developed a decomposition algorithm for solving th...

Charged rotating black holes of Einstein-Maxwell-Chern-Simons theory in odd dimensions, D ≥ 5, may possess a negative horizon mass, while their total mass is positive. This surprising feature is related to the existence of counterrotating solutions, where the horizon angular velocity Ω and the angular momentum J possess opposite signs. Black holes may further possess vanishing horizon angular v...

We consider infinite horizon production scheduling under stochastic demand. All problem data are allowed to vary across periods, including demand distributions, costs, and revenues. A forecast horizon, when it exists, is a finite problem horizon with the property that the corresponding first-period optimal production decision remains optimal regardless of demand and cost projections beyond this...