This paper presents the first actor-critic algorithm for off-policy reinforcement learning. Our algorithm is online and incremental, and its per-time-step complexity scales linearly with the number of learned weights. Previous work on actor-critic algorithms is limited to the on-policy setting and does not take advantage of the recent advances in offpolicy gradient temporal-difference learning....

We propose a new algorithm, Mean Actor-Critic (MAC), for discrete-action continuous-state reinforcement learning. MAC is a policy gradient algorithm that uses the agent’s explicit representation of all action values to estimate the gradient of the policy, rather than using only the actions that were actually executed. This significantly reduces variance in the gradient updates and removes the n...

Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks. It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific. We propose an algorithm to automatically learn lear...

This paper describes an intelligent computer-aided architectural design system (ICAAD) called ICADS. ICADS encapsulates different types of design knowledge into independent “critic” modules. Each “critic” module possesses expertise in evaluating an architect’s work in different areas of architectural design and can offer expert advice when needed. This research focuses on the representation of ...

Neural networks have been successfully used for implementing control architectures for different applications. In this work, we examine a neural network augmented adaptive critic as a Level 2 intelligent controller for a C-17 aircraft. This intelligent control architecture utilizes an adaptive critic to tune the parameters of a reference model, which is then used to define the angular rate comm...

Ann Jurecic is assistant professor in the Department of English at Rutgers University, New Brunswick, where she teaches courses in writing as well as literature and medicine. Her book Illness as Narrative is forthcoming from University of Pittsburgh Press. Her work has appeared in the journals Literature and Medicine, Pedagogy, and Biography as well as in College English, and she serves as book...