2021

Tianshou: a Highly Modularized Deep Reinforcement Learning Library

Weng, Jiayi, Chen, Huayu, Yan, Dong et al.

Understand

In this paper, we present Tianshou, a highly modularized Python library for deep reinforcement learning (DRL) that uses PyTorch as its backend.

Tianshou intends to be research-friendly by providing a flexible and reliable infrastructure of DRL algorithms.
It supports online and offline training with more than 20 classic algorithms through a unified interface.
To facilitate related research and prove Tianshou's reliability, we have released Tianshou's benchmark of MuJoCo environments, covering eight classic algorithms with state-of-the-art performance.

MuJoCo: A physics engine for model-based control
Emanuel Todorov, Tom Erez, and Yuval Tassa · 2012
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al · 2015
Earlier work this paper cites.
OpenAI Gym
Original
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba · 2016
Earlier work this paper cites.
Benchmarking deep reinforcement learning for continuous control
Yan Duan, Xi Chen, Rein Houthooft, John Schulman, and Pieter Abbeel · 2016
Earlier work this paper cites.
Generative adversarial imitation learning
Jonathan Ho and Stefano Ermon · 2016
Earlier work this paper cites.
keras-rl
Matthias Plappert · 2016
Earlier work this paper cites.
High-dimensional continuous control using generalized advantage estimation
John Schulman, Philipp Moritz, Sergey Levine, Michael I. Jordan, and Pieter Abbeel · 2016
Earlier work this paper cites.

Learning values across many orders of magnitude
Hado P van Hasselt, Arthur Guez, Matteo Hessel, Volodymyr Mnih, and David Silver · 2016
Cited alongside, same era.
Reinforcement learning Coach
Itai Caspi, Gal Leibovich, Gal Novik, and Shadi Endrawis · 2017
Cited alongside, same era.
Efficient parallel methods for deep reinforcement learning
Original
Alfredo V Clemente, Humberto N Castejón, and Arjun Chandra · 2017
Cited alongside, same era.
Tensorforce: a tensorflow library for applied reinforcement learning
Alexander Kuhnle, Michael Schaarschmidt, and Kai Fricke · 2017
Cited alongside, same era.
Curiosity-driven exploration by self-supervised prediction
Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell · 2017
Cited alongside, same era.
Mastering the game of go without human knowledge
David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al · 2017
Cited alongside, same era.
RLlib: Abstractions for distributed reinforcement learning
Eric Liang, Richard Liaw, Robert Nishihara, Philipp Moritz, Roy Fox, Ken Goldberg, Joseph Gonzalez, Michael I. Jordan, and Ion Stoica · 2018
Cited alongside, same era.

Time limits in reinforcement learning
Fabio Pardo, Arash Tavakoli, Vitaly Levdik, and Petar Kormushev · 2018
Later among the works it cites.
rlpyt: A research code base for deep reinforcement learning in pytorch
Original
Adam Stooke and Pieter Abbeel · 2019
Later among the works it cites.
MushroomRL: Simplifying reinforcement learning research
Original
Carlo D’Eramo, Davide Tateo, Andrea Bonarini, Marcello Restelli, and Jan Peters · 2020
Later among the works it cites.
ChainerRL: A deep reinforcement learning library
Yasuhiro Fujita, Prabhat Nagarajan, Toshiki Kataoka, and Takahiro Ishikawa · 2021
Closest in time.
Stable-Baselines3: Reliable reinforcement learning implementations
Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann · 2021
Closest in time.
d3rlpy: An offline deep reinforcement library
Michita Imai Takuma Seno · 2021
Closest in time.
EnvPool: A highly parallel reinforcement learning environment execution engine
Original
Jiayi Weng, Min Lin, Shengyi Huang, Bo Liu, Denys Makoviichuk, Viktor Makoviychuk, Zichen Liu, Yufan Song, Ting Luo, Yukun Jiang, Zhongwen Xu, and Shuicheng Yan · 2022
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…