A new fast algorithm for reinforcement learning (RL) in continuous state and action spaces is proposed. Unlike algorithms based on dynamic programming, the proposed algorithm uses neither temporal nor spatial diierencing. Instead, it couples the solution of the Hamilton-Jacobi-Bellman (HJB) partial diierential equation with the structure of the function approximators that are commonly used toge...

Water management in agriculture is a key issue due to the increasing problem of water scarcity worldwide. Based on the recent progress in the dynamic modeling of plant growth in interaction with the water resource, our objective is to study the optimal control problem of crop irrigation. For this purpose, we first describe the LNAS model for sugar beet growth, driving the dynamics of both plant...

We consider the problem of maximizing expected utility for a power investor who can allocate his wealth in a stock, a defaultable security, and a money market account. The dynamics of these security prices are governed by geometric Brownian motions modulated by a hidden continuous time finite state Markov chain. We reduce the partially observed stochastic control problem to a complete observati...

Many nonlinear option pricing problems can be formulated as optimal control problems, leading to Hamilton-Jacobi-Bellman (HJB) or Hamilton-Jacobi-Bellman-Isaacs (HJBI) equations. We show that such formulations are very convenient for developing monotone discretization methods which ensure convergence to the financially relevant solution, which in this case is the viscosity solution. In addition...

Hysteresis in smart materials hinders their wider applicability in actuators. The low dimensional hysteresis models for these materials are hybrid systems with both controlled switching and autonomous switching. In particular, they belong to the class of Duhem hysteresis models and can be formulated as systems with both continuous and switching controls. In this paper, we study the control meth...

This paper is concerned with mean-variance portfolio selection problems in continuoustime under the constraint that short-selling of stocks is prohibited. The problem is formulated as a stochastic optimal linear-quadratic (LQ) control problem. However, this LQ problem is not a conventional one in that the control (portfolio) is constrained to take nonnegative values due to the no-shorting restr...

This paper proposes a study of Reinforcement Learning (RL) for continuous state-space and time control problems, based on the theoretical framework of viscosity solutions (VSs). We use the method of dynamic programming (DP) which introduces the value function (VF), expectationof the best future cumulativereinforcement. In the continuous case, the value function satisses a non-linear rst (or sec...

Exposure of perfused rat lungs to lipopolysaccharides (LPS) causes induction of cyclooxygenase-2 followed by thromboxane (TX)-mediated bronchoconstriction (BC). Recently, phosphodiesterase (PDE) inhibitors have received much interest because they not only are bronchodilators but also can suppress release of proinflammatory mediators. In the present study, we investigated the effect of three dif...

The rod photoreceptor phosphodiesterase (PDE) is unique among all known vertebrate PDE families for several reasons. It is a catalytic heterodimer (alphabeta); it is directly activated by a G-protein, transducin; and its active sites are regulated by inhibitory gamma subunits. Rod PDE binds cGMP at two noncatalytic sites on the alphabeta dimer, but their function is unclear. We show that transd...

Recent studies have demonstrated that functionally discrete pools of phosphodiesterase (PDE) activity regulate distinct cellular functions. While the importance of localized pools of enzyme activity has become apparent, few studies have estimated enzyme activity within discrete subcellular compartments. Here we present an approach to estimate near-membrane PDE activity. First, total PDE activit...