We propose a new probabilistic numerical scheme for fully nonlinear equation of Hamilton-Jacobi-Bellman (HJB) type associated to stochastic control problem, which is based on the Feynman-Kac representation in [13] by means of control randomization and backward stochastic differential equation with nonpositive jumps. We study a discrete time approximation for the minimal solution to this class o...

Reinforcement learning methods for discrete and semi-Markov decision problems such as Real-Time Dynamic Programming can be generalized for Controlled Diiusion Processes. The optimal control problem reduces to a boundary value problem for a fully nonlinear second-order elliptic diierential equation of Hamilton-Jacobi-Bellman (HJB-) type. Numerical analysis provides multi-grid methods for this ki...

A sufficient condition to solve an optimal control problem is to solve the Hamilton–Jacobi–Bellman (HJB) equation. However, finding a value function that satisfies the HJB equation for a nonlinear system is challenging. For an optimal control problem when a cost function is provided a priori, previous efforts have utilized feedback linearization methods which assume exact model knowledge, or ha...

Abstract We study a second order Backward Differentiation Formula (BDF) scheme for the numerical approximation of linear parabolic equations and nonlinear Hamilton–Jacobi–Bellman (HJB) equations. The lack monotonicity BDF prevents use well-known convergence results solutions in viscosity sense. first consider one-dimensional uniformly prove stability with respect to perturbations, $$L^2$$ <mml:...

In this chapter erosion is generalized to the space of diffusion weighted MRI data. This is done effectively by solving a Hamilton-Jacobi-Bellman (HJB) system (erosion) on the coupled space of three dimensional positions and orientations, embedded as a quotient in the group of three dimensional rigid body motions. The solution to the HJB equations is given by a well-posed morphological convolut...

5 The no arbitrage pricing of Guaranteed Minimum Withdrawal Benefits (GMWB) contracts 6 results in a singular stochastic control problem which can be formulated as a Hamilton Jacobi 7 Bellman (HJB) Variational Inequality (VI). Recently, a penalty method has been suggested for 8 solution of this HJB variational inequality (Dai et al., 2008). This method is very simple to 9 implement. In this art...

We study the exploratory Hamilton–Jacobi–Bellman (HJB) equation arising from entropy-regularized control problem, which was formulated by Wang, Zariphopoulou, and Zhou (J. Mach. Learn. Res., 21 (2020), 198) in context of reinforcement learning continuous time space. establish well-posedness regularity viscosity solution to equation, as well convergence problem classical stochastic when level ex...

Using the “basic monotonicity property” along locally admissible trajectories, we extend to very general problems certain existing results concerning the differential inequalities verified by the value function of an optimal control problem; these differential inequalities are expressed in terms of its contingent, quasitangent, and peritangent (Clarke’s) directional derivatives and in terms of ...

Abstract. In this paper, the solvability of a class of forward-backward stochastic di erential equations (SDEs for short) over an arbitrarily prescribed time duration is studied. We design a stochastic relaxed control problem, with both drift and di usion all being controlled, so that the solvability problem is converted to a problem of nding the nodal set of the viscosity solution to a certain...

This paper deals with the optimal control of a stochastic delay differential equation arising in the management of a pension fund with surplus. The problem is approached by the tool of the representation in infinite dimension. We show the equivalence between the 1-dimensional delay problem and the associated infinite dimensional problem without delay. Then we prove that the value function is co...