Detect, Understand, Act: A Neuro-symbolic Hierarchical Reinforcement Learning Framework

Abstract In this paper we introduce Detect, Understand, Act (DUA), a neuro-symbolic reinforcement learning framework. The Detect component is composed of traditional computer vision object detector and tracker. houses set options, high-level actions enacted by pre-trained deep (DRL) policies. Understand provides novel answer programming (ASP) paradigm for symbolically implementing meta-policy over options effectively it using inductive logic (ILP). We evaluate our framework on the Animal-AI (AAI) competition testbed, physical cognitive reasoning problems. Given DRL policies, DUA requires only few examples to learn that allows improve state-of-the-art multiple most challenging categories from testbed. constitutes first holistic hybrid integration vision, ILP applied an AAI-like environment sets foundations further use in complex challenges.