We consider a continuous-time stochastic optimization problem with infinite horizon, linear dynamics, and cone constraintswhich includes as a particular case portfolio selection problems under transaction costs for models of stock and currency markets. Using an appropriate geometric formalism we show that the Bellman function is the unique viscosity solution of a HJB equation.

This paper presents a novel approach for developing tracking controllers for nonlinear systems. The approach involves the numerical solution, by Galerkin approximation , of the time-varying Hamilton-Jacobi-Bellman (HJB) equation and results in a nonlinear controller approximating the optimal tracking control law for a spec-iied desired trajectory and cost function. Experimental results are pres...

The purpose of this paper is to describe the application of the notion of viscosity solutions to solve the Hamilton–Jacobi–Bellman (HJB) equation associated with an important class of optimal control problems for quantum spin systems. The HJB equation that arises in the control problems of interest is a first-order nonlinear partial differential equation defined on a Lie group. Hence we employ ...

We consider the numerical solution of discretised Hamilton-Jacobi-Bellman (HJB) equations with applications in finance. For the discrete linear complementarity problem arising in American option pricing, we study a policy iteration method. We show, analytically and numerically, that, in standard situations, the computational cost of this approach is comparable to that of European option pricing...

In statistical control, the cost function is viewed as a random variable and one optimizes the distribution of the cost function through the cost cumulants. We consider a statistical control problem for a control-affine nonlinear system with a nonquadratic cost function. Using the Dynkin formula, the Hamilton–Jacobi–Bellman equation for the nth cost moment case is derived as a necessary conditi...

This paper presents an application of the nonlinear optimal control techniques to the design of launch vehicle autopilots. The optimal control is given by the solution to the Hamilton-Jacobi-Bellman (HJB) equation, which in this case cannot be solved explicity. A method based upon Successive Galerkin Approximation (SGA), is used to obtain an approximate optimal solution. Simulation results invo...

This paper introduces an optimal tracking controller for robot manipulators with saturation torques. The model is presented as a strict-feedback nonlinear system. Firstly, the position control problem transformed into problem. Subsequently, saturated law designed. determined through solution of Hamilton-Jacobi-Bellman (HJB) equation. We use reinforcement learning algorithm only one neural netwo...

We prove the convergence of a non-monotonous scheme for a one-dimensional first order Hamilton-Jacobi-Bellman equation of the form vt+maxα(f(x, α)vx) = 0, v(0, x) = v0(x). The scheme is related to the HJB-UltraBee scheme suggested in [7]. We show for general discontinuous initial data a first-order convergence of the scheme, in L-norm, towards the viscosity solution. We also illustrate the non-...

The Brownian motion in equation (1) is defined on a complete probability space and generates a filtration F . Throughout this appendix, “adapted process” means F (t) adapted. Lemma 1. Let me scale all variables by output. Then, given the conjecture that the saver’s value function only depends on scaled deposits and scaled net transfers, U(D̃,Π̃), the optimization problem is solved by the followin...

A novel approach for online design of optimal control systems based on QRtuning, state and action-dependent heuristic dynamic programming, and approximate-LS solutions of the Hamilton-Jacobi-Bellman (HJB) equation is the main concern of this paper. The QR-tuning for optimal control systems takes into account heuristic variations in the weighting matrices Q and R of the discrete linear quadratic...