Transfer Reinforcement Learning
نویسندگان
چکیده
The objective of transfer reinforcement learning is to generalize from a set of previous tasks to unseen new tasks. In this work, we focus on the transfer scenario where the dynamics among tasks are the same, but their goals differ. Although general value function (Sutton et al., 2011) has been shown to be useful for knowledge transfer, learning a universal value function can be challenging in practice. To attack this, we propose (1) to use universal successor representations (USR) to represent the transferable knowledge and (2) a USR approximator (USRA) that can be trained by interacting with the environment. Our experiments show that USR can be effectively applied to new tasks, and the agent initialized by the trained USRA can achieve the goal considerably faster than random initialization.
منابع مشابه
Hierarchical Functional Concepts for Knowledge Transfer among Reinforcement Learning Agents
This article introduces the notions of functional space and concept as a way of knowledge representation and abstraction for Reinforcement Learning agents. These definitions are used as a tool of knowledge transfer among agents. The agents are assumed to be heterogeneous; they have different state spaces but share a same dynamic, reward and action space. In other words, the agents are assumed t...
متن کاملFixed vs Dynamic Sub-transfer in Reinforcement Learning Technical report
We survey various transfer methods in Q-learning, a type of reinforcement learning, and present a variation on fixed sub-transfer which we call dynamic sub-transfer. We describe the pros and cons of dynamic sub-transfer as compared with the other transfer methods, and we describe qualitatively the situations where this method would be preferred over the fixed version of sub-transfer.
متن کاملTransfer Learning in Multi-Agent Reinforcement Learning Domains
Transfer learning refers to the process of reusing knowledge from past tasks in order to speed up the learning procedure in new tasks. In reinforcement learning, where agents often require a considerable amount of training, transfer learning comprises a suitable solution for speeding up learning. Transfer learning methods have primarily been applied in single-agent reinforcement learning algori...
متن کاملReinforcement Learning Task Clustering (RLTC)
This work represents the first step towards a task library system in the reinforcement learning domain. Task libraries could be useful in speeding up the learning of new tasks through task transfer. Related transfer can increase learning rate and can help prevent convergence to sub-optimal policies in reinforcement learning. Unrelated transfer can be extremely detrimental to the learning rate. ...
متن کاملTransfer Learning for Reinforcement Learning Domains: A Survey
The reinforcement learning paradigm is a popular way to address problems that have only limited environmental feedback, rather than correctly labeled examples, as is common in other machine learning contexts. While significant progress has been made to improve learning in a single task, the idea of transfer learning has only recently been applied to reinforcement learning tasks. The core idea o...
متن کاملTransfer Learning Method Using Ontology for Heterogeneous Multi-agent Reinforcement Learning
This paper presents a framework, called the knowledge co-creation framework (KCF), for heterogeneous multiagent robot systems that use a transfer learning method. A multiagent robot system (MARS) that utilizes reinforcement learning and a transfer learning method has recently been studied in realworld situations. In MARS, autonomous agents obtain behavior autonomously through multi-agent reinfo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2018