Efficient Reinforcement Learning Experimentation in PyTorch

Abstract: Deep reinforcement learning (RL) has proved successful at solving challenging environments but often requires long training times and very many samples. Furthermore, advancing artificial intelligence requires to easily prototype new methods, yet avoiding impractically slow experimental turnaround times. To this end, we present a PyTorch-based library for RL with a modular design that allows composing agents based on three components types: actors, storages and algorithms. Additionally, the definition of synchronous and asynchronous architectures is permitted with flexibility and independence of the components. We present several standard use-cases of the library and showcase its potential by obtaining the highest to-date test performance on the Obstacle Tower Unity3D challenge environment. In summary, we believe that this work helps accelerate experimentation of new ideas, simplifying research and enabling to tackle more challenging RL problems.