Provably Efficient Reinforcement Learning with Linear Function Approximation

Modern reinforcement learning (RL) is commonly applied to practical problems with an enormous number of states, where function approximation must be deployed approximate either the value or policy. The introduction raises a fundamental set challenges involving computational and statistical efficiency, especially given need manage exploration/exploitation trade-off. As result, core RL question remains open: how can we design provably efficient algorithms that incorporate approximation? This persists even in basic setting linear dynamics rewards, for which only needed. paper presents first provable algorithm both polynomial run time sample complexity this setting, without requiring “simulator” additional assumptions. Concretely, prove optimistic modification least-squares iteration—a classical frequently studied setting—achieves [Formula: see text] regret, d ambient dimension feature space, H length each episode, T total steps. Importantly, such regret independent states actions. Funding: work was supported by Defense Advanced Research Projects Agency program on Lifelong Learning Machines.