Hierarchical reinforcement learning (HRL) proposes to solve difficult tasks by performing decision-making and control at successively higher levels of temporal abstraction. However, off-policy HRL often suffers from the problem a non-stationary high-level policy since low-level is constantly changing. In this paper, we propose novel approach for mitigating non-stationarity adversarially enforci...
