We consider optimal control problems for systems described by stochastic differential equations with delay. We state conditions for certain classes of such systems under which the stochastic control problems become finite-dimensional. These conditions are illustrated with three applications. First, we solve some linear quadratic problems with delay. Then we find the optimal consumption rate in ...

We consider a stochastic control model in which an economic unit has productive capital and also liabilities in the form of debt. The worth of capital changes over time through investment, and also through random Brownian fluctuations in the unit price of capital. Income from production is also subject to random Brownian fluctuations. The goal is to choose investment and consumption controls wh...

We consider a stochastic control model driven by a fractional Brownian motion. This model is a formal approximation to a queueing network with an ON–OFF input process. We study stochastic control problems associated with the long-run average cost, the infinite-horizondiscounted cost, and thefinite-horizon cost. In addition, wefind a solution to a constrained minimization problem as an applicati...

A stochastic control model is proposed as a paradigm for the design of optimal timing of greenhouse gases (GHG) emissions abatement. The resolution of uncertainty concerning climate sensitivity and the technological breakthrough providing access to a carbon-free production economy are modeled as controlled stochastic jump processes. The optimal policy is characterized using the dynamic programm...

We consider a stochastic control problem where the set of controls is not necessarily convex and the system is governed by a nonlinear backward stochastic differential equation. We establish necessary as well as sufficient conditions of optimality for two models. The first concerns the strict (classical) controls. The second is an extension of the first to relaxed controls, who are a measure va...

This paper considers robust reduced order H∞ observer-based control for stochastic systems. The approach is divided into two steps. First, we search for a linear control law to ensure H∞ specification. Then, the obtained linear combination of the state is used to solve an “unbiasedness” (decoupling) condition on the drift part of the closed-loop system. Finally, deterministic functional filteri...

We develop the method of stochastic modified equations (SME), in which stochastic gradient algorithms are approximated in the weak sense by continuous-time stochastic differential equations. We exploit the continuous formulation together with optimal control theory to derive novel adaptive hyper-parameter adjustment policies. Our algorithms have competitive performance with the added benefit of...

Control theory is a mathematical description of how to act optimally to gain future rewards. In this paper I give an introduction to deterministic and stochastic control theory; partial observability, learning and the combined problem of inference and control. Subsequently, I discuss a class of non-linear stochastic control problems that can be efficiently solved using a path integral. In this ...

In this paper, we study the problem of estimating a Markov chain X(signal) from its noisy partial information Y , when the transition probability kernel depends on some unknown parameters. Our goal is to compute the conditional distribution process P{Xn|Yn, . . . , Y1}, referred to hereafter as the optimal filter. Following a standard Bayesian technique, we treat the parameters as a nondynamic ...