An RL Approach to Coordinate Exploration with Limited Communication in Continuous Action Games

Learning automata are reinforcement learners belonging to the category of policy iterators. They have already been shown to exhibit nice convergence properties in discrete action games. Recently, a new formulation for a Continuous Action Reinforcement Learning Automaton (CARLA) was proposed. In this paper we study the behavior of these CARLA in continuous action games and propose a novel method for coordinated exploration of the joint-action space. Our method allows a team of independent learners, using CARLA, to find the optimal joint action in common interest settings. We first show that independent agents using CARLA will converge to a local optimum of the continuous action game. I addition we introduce a method for coordinated exploration which allows the team of agents to find the global optimum of the game. We also validate our approach in a number of experiments.