We consider one of the fundamental limitations of indirect adaptive control based on the minimization of a quadratic cost criterion and the certainty equivalence principle. We show that the interaction between (closed-loop) identification and optimal control is conflictive in the sense that almost all possible limits of the sequence of parameter estimates induce suboptimal behavior of the adapt...

In this paper we solve a finite-horizon partially observed risk-sensitive stochastic optimal control problem for discrete-time nonlinear systems and obtain small noise and small risk limits. The small noise limit is interpreted as a deterministic partially observed dynamic game, and new insights into the optimal solution of such game problems are obtained. Both the risk-sensitive stochastic con...

Abstract. We consider solutions of the equation −Δu + λu + |∇u|q = f , which blow up uniformly at the boundary of a smooth domain, that can be interpreted as the value function of a state constraint control problem for a Brownian motion. We prove a complete asymptotic expansion of the gradient at the boundary, giving the precise behavior of normal and tangent components. The result is achieved ...

This paper approaches optimal control problems for discrete-time controlled Markov processes by representing the value of the problem in a dual Lagrangian form; the value is expressed as an infimum over a family of Lagrangian martingales of an expectation of a pathwise supremum of the objective adjusted by the Lagrangian martingale term. This representation opens up the possibility of numerical...

We formulate and solve a problem that combines the features of the so-called monotone follower of singular stochastic control theory with optimal stopping. In particular, we consider a stochastic system whose uncontrolled state dynamics are modelled by a general one-dimensional Itô diffusion. The aim of the problem that we solve is to maximise the utility derived from the system’s state at the ...

We study quadratic optimal stochastic control problems with control dependent noise state equation perturbed by an affine term and with stochastic coefficients. Both infinite horizon case and ergodic case are treated. To this purpose we introduce a Backward Stochastic Riccati Equation and a dual backward stochastic equation, both considered in the whole time line. Besides some stabilizability c...

While the current robust nonlinear control toolbox includes a number of methods for systems aane in a deterministic bounded disturbances, the problem when the disturbance is unbounded stochastic noise has hardly been considered. We present a control design which achieves global asymptotic (Lyapunov) stability in probability for a class of strict-feedback nonlinear continuous-time systems driven...

The optimal control of nonlinear stochastic systems is considered in this paper. The central role played by the Fokker-Planck-Kolmogorov equation in the stochastic control problem is shown under the assumption of asymptotic stability. A computational approach for the problem is devised based on policy iteration/ successive approximations, and a finite dimensional approximation of the control pa...

This article is a survey of the early development of selected areas in nonlinear continuous-time stochastic control. Key developments in optimal control and the dynamic programming principle, existence of optimal controls under complete and partial observations, nonlinear filtering, stochastic stability, the stochastic maximum principle and ergodic control are discussed. Issues concerning wide ...

We consider damped stochastic systems in a controlled (time-varying) quadratic potential and study their transition between specified Gibbs-equilibria states in finite time. By the second law of thermodynamics, the minimum amount of work needed to transition from one equilibrium state to another is the difference between the Helmholtz free energy of the two states and can only be achieved by a ...