This paper proposes a new actor-critic-style algorithm called Dual Actor-Critic or Dual-AC. It is derived in a principled way from the Lagrangian dual form of the Bellman optimality equation, which can be viewed as a two-player game between the actor and a critic-like function, which is named as dual critic. Compared to its actor-critic relatives, Dual-AC has the desired property that the actor...

The highest-order function of the mind is as a theorist. The memory system accumulates information about the outside world. The mind’s theorist must sort through the information to formulate a theory about that world. The basic component of the system for theory building is a process called trolling. When the conscious mind is not being bombarded by external stimuli, or during certain stages of...

In earlier papers a critic for automatically generalizing conjectures in the context of failed inductive proofs was presented. The critic exploits the partial success of the search control heuristic known as rippling. Through empirical testing a natural generalization and extension of the basic critic emerged. Here we describe our extended generalization critic together with some promising expe...

We prove the convergence of an actor/critic algorithm that is equivalent to Q-learning by construction. Its equivalence is achieved by encoding Q-values within the policy and value function of the actor and critic. The resultant actor/critic algorithm is novel in two ways: it updates the critic only when the most probable action is executed from any given state, and it rewards the actor using c...

In this issue of JET, we celebrate Dave Cass’s remarkable scientific career with two symposia: (1) “Economic Growth” organized by Daron Acemoglu, and (2) “General Equilibrium” organized by Yves Balasko and John Geanakoplos. Dividing the issue into two parts should not obscure the fact that Dave recognized only one economic science, one that includes growth, general equilibrium (GE), and more. D...

Along with Marx, Friedrich Nietzsche can be read as a great theorist and critic of modernity who carried out a "ruthless criticism of all that exists" (Marx 1975c [1843]: 142). His powerful broadsides against religion, morality, and philosophy deploy a mixture of Enlightenment-inspired criticism and anti-Enlightenment vitalism to attack the life-negating aspects of modern culture. In addition, ...

Actor-critic methods solve reinforcement learning problems by updating a parameterized policy known as an actor in a direction that increases an estimate of the expected return known as a critic. However, existing actor-critic methods only use values or gradients of the critic to update the policy parameter. In this paper, we propose a novel actor-critic method called the guide actor-critic (GA...