Paper

A Survey of Zero-shot Generalisation in Deep Reinforcement Learning

Kirk, Robert, Zhang, Amy, Grefenstette, Edward et al.

Understand

The study of zero-shot generalisation (ZSG) in deep Reinforcement Learning (RL) aims to produce RL algorithms whose policies generalise well to novel unseen situations at deployment time, avoiding overfitting to their training environments.

Tackling this is vital if we are to deploy reinforcement learning algorithms in real world scenarios, where the environment will be diverse, dynamic and unpredictable.
This survey is an overview of this nascent field.
We rely on a unifying formalism and terminology for discussing different ZSG problems, building upon previous works.

Built on

Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions.
Original
Wang, R., Lehman, J., Clune, J., and Stanley, K. O. (2019) · 1901
Earlier work this paper cites.
Investigating Generalisation in Continuous Deep Reinforcement Learning.
Original
Zhao, C., Sigaud, O., Stulp, F., and Hospedales, T. M. (2019) · 1902
Earlier work this paper cites.
An empirical investigation of the challenges of real-world reinforcement learning.
Original
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., and Hester, T. (2021) · 1904
Earlier work this paper cites.
Continual Reinforcement Learning in 3D Non-stationary Environments.
Original
Lomonaco, V., Desai, K., Culurciello, E., and Maltoni, D. (2020) · 1905
Earlier work this paper cites.
Toybox: A Suite of Environments for Experimental Evaluation of Deep Reinforcement Learning.
Original
Tosch, E., Clary, K., Foley, J., and Jensen, D. (2019) · 1905
Earlier work this paper cites.
Wasserstein Robust Reinforcement Learning.
Original
Abdullah, M. A., Ren, H., Ammar, H. B., Milenkovic, V., Luo, R., Zhang, M., and Wang, J. (2019) · 1907
Earlier work this paper cites.
Invariant Risk Minimization.
Original
Arjovsky, M., Bottou, L., Gulrajani, I., and Lopez-Paz, D. (2020) · 1907
Earlier work this paper cites.
Reasoning and Generalization in RL: A Tool Use Perspective.
Original
Wenke, S., Saunders, D., Qiu, M., and Fleming, J. (2019) · 1907
Earlier work this paper cites.
Compositionality decomposed: How do neural networks generalise?.
Original
Hupkes, D., Dankers, V., Mul, M., and Bruni, E. (2020) · 1908
Earlier work this paper cites.
Generalization in Transfer Learning.
Original
Ada, S. E., Ugur, E., and Akin, H. L. (2021) · 1909
Earlier work this paper cites.
RLBench: The Robot Learning Benchmark & Learning Environment.
Original
James, S., Ma, Z., Arrojo, D. R., and Davison, A. J. (2019a) · 1909
Earlier work this paper cites.
Solving Rubik’s Cube with a Robot Hand.
Original
OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., Ribas, R., Schneider, J., Tezak, N., Tworek, J., Welinder, P., Weng, L., Yuan, Q., Zaremba, W., and Zhang, L. (2019a) · 1910
Earlier work this paper cites.
Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning.
Original
Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S. (2019) · 1910
Earlier work this paper cites.
Causality for Machine Learning.
Original
Schölkopf, B. (2019) · 1911
Earlier work this paper cites.
Dota 2 with Large Scale Deep Reinforcement Learning.
Original
OpenAI, Berner, C., Brockman, G., Chan, B., Cheung, V., Dębiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., Józefowicz, R., Gray, S., Olsson, C., Pachocki, J., Petrov, M., Pinto, H. P. d. O., Raiman, J., Salimans, T., Schlatter, J., Schneider, J., Sidor, S., Sutskever, I., Tang, J., Wolski, F., and Zhang, S. (2019b) · 1912
Earlier work this paper cites.
Stochastic Games*.
Shapley, L. S. (1953) · 1953
Earlier work this paper cites.
No free lunch theorems for optimization.
Wolpert, D., and Macready, W. (1997) · 1997
Earlier work this paper cites.
Stochastic dynamic programming with factored representations.
Boutilier, C., Dearden, R., and Goldszmidt, M. (2000) · 2000
Earlier work this paper cites.
Robust reinforcement learning
Morimoto, J., and Doya, K. (2000) · 2000
Earlier work this paper cites.
Algorithms for inverse reinforcement learning
Ng, A. Y., and Russell, S. J. (2000) · 2000
Earlier work this paper cites.
Improving Generalization of Reinforcement Learning with Minimax Distributional Soft Actor-Critic.
Original
Ren, Y., Duan, J., Li, S. E., Guan, Y., and Sun, Q. (2020) · 2002
Earlier work this paper cites.
Solving relational MDPs with first-order machine learning
Mausam, D. S. W. (2003) · 2003
Earlier work this paper cites.
Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey.
Original
Narvekar, S., Peng, B., Leonetti, M., Sinapov, J., Taylor, M. E., and Stone, P. (2020) · 2003
Earlier work this paper cites.
Multi-Task Reinforcement Learning with Soft Modularization.
Original
Yang, R., Xu, H., Wu, Y., and Wang, X. (2020a) · 2003
Earlier work this paper cites.
D4RL: Datasets for Deep Data-Driven Reinforcement Learning.
Original
Fu, J., Kumar, A., Nachum, O., Tucker, G., and Levine, S. (2021) · 2004
Earlier work this paper cites.
Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels.
Original
Kostrikov, I., Yarats, D., and Fergus, R. (2021) · 2004
Earlier work this paper cites.
Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Text.
Original
Hill, F., Mokra, S., Wong, N., and Harley, T. (2020b) · 2005
Earlier work this paper cites.
Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems.
Original
Levine, S., Kumar, A., Tucker, G., and Fu, J. (2020) · 2005
Earlier work this paper cites.
RL Unplugged: A Suite of Benchmarks for Offline Reinforcement Learning.
Original
Gulcehre, C., Wang, Z., Novikov, A., Paine, T. L., Colmenarejo, S. G., Zolna, K., Agarwal, R., Merel, J., Mankowitz, D., Paduraru, C., Dulac-Arnold, G., Li, J., Norouzi, M., Hoffman, M., Nachum, O., Tucker, G., Heess, N., and de Freitas, N. (2021) · 2006
Earlier work this paper cites.
Transient Non-Stationarity and Generalisation in Deep Reinforcement Learning.
Original
Igl, M., Farquhar, G., Luketina, J., Boehmer, W., and Whiteson, S. (2021) · 2006
Earlier work this paper cites.
AWAC: Accelerating Online Reinforcement Learning with Offline Datasets.
Original
Nair, A., Gupta, A., Dalal, M., and Levine, S. (2021) · 2006
Earlier work this paper cites.
Automatic Data Augmentation for Generalization in Deep Reinforcement Learning.
Original
Raileanu, R., Goldstein, M., Yarats, D., Kostrikov, I., and Fergus, R. (2021) · 2006
Earlier work this paper cites.
Invariant Policy Optimization: Towards Stronger Generalization in Reinforcement Learning.
Original
Sonar, A., Pacelli, V., and Majumdar, A. (2020) · 2006
Earlier work this paper cites.
Efficient structure learning in factored-state MDPs
Strehl, A. L., Diuk, C., and Littman, M. L. (2007) · 2007
Earlier work this paper cites.
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., and Littman, M. L. (2008) · 2008
Earlier work this paper cites.
Offline Meta Learning of Exploration.
Original
Dorfman, R., Shenfeld, I., and Tamar, A. (2021) · 2008
Earlier work this paper cites.
Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning.
Original
Lu, X., Lee, K., Abbeel, P., and Tiomkin, S. (2020) · 2008
Earlier work this paper cites.
Phasic Policy Gradient.
Original
Cobbe, K., Hilton, J., Klimov, O., and Schulman, J. (2020b) · 2009
Earlier work this paper cites.
Robosuite: A Modular Simulation Framework and Benchmark for Robot Learning.
Original
Zhu, Y., Wong, J., Mandlekar, A., and Martín-Martín, R. (2020) · 2009
Earlier work this paper cites.
Transfer Learning in Deep Reinforcement Learning: A Survey.
Original
Zhu, Z., Lin, K., and Zhou, J. (2021) · 2009
Earlier work this paper cites.
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning.
Original
Ahmed, O., Träuble, F., Goyal, A., Neitz, A., Bengio, Y., Schölkopf, B., Wüthrich, M., and Bauer, S. (2020) · 2010
Earlier work this paper cites.
Measuring Visual Generalization in Continuous Control from Pixels.
Original
Grigsby, J., and Qi, Y. (2020) · 2010
Earlier work this paper cites.
Prioritized Level Replay.
Original
Jiang, M., Grefenstette, E., and Rocktäschel, T. (2021b) · 2010
Earlier work this paper cites.
Trajectory-wise Multiple Choice Learning for Dynamics Generalization in Reinforcement Learning.
Original
Seo, Y., Lee, K., Clavera, I., Kurutach, T., Shin, J., and Abbeel, P. (2020) · 2010
Earlier work this paper cites.
Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning.
Original
Chen, V., Gupta, A., and Marino, K. (2021) · 2011
Earlier work this paper cites.
Reinforcement Learning with Videos: Combining Offline Observations with Interaction.
Original
Schmeckpeper, K., Rybkin, O., Daniilidis, K., Levine, S., and Finn, C. (2020) · 2011
Earlier work this paper cites.
Protecting against evaluation overfitting in empirical reinforcement learning
Whiteson, S., Tanner, B., Taylor, M. E., and Stone, P. (2011) · 2011
Earlier work this paper cites.
MuJoCo: A physics engine for model-based control
E. Todorov, T. Erez, and Y. Tassa (2012) · 2012
Earlier work this paper cites.
Towards Continual Reinforcement Learning: A Review and Perspectives.
Original
Khetarpal, K., Riemer, M., Rish, I., and Precup, D. (2020) · 2012
Earlier work this paper cites.
Demonstration-efficient Inverse Reinforcement Learning in Procedurally Generated Environments.
Original
Sestini, A., Kuhnle, A., and Bagdanov, A. D. (2020) · 2012
Earlier work this paper cites.
iGibson 1.0: A Simulation Environment for Interactive Tasks in Large Realistic Scenes.
Original
Shen, B., Xia, F., Li, C., Martín-Martín, R., Fan, L., Wang, G., Pérez-D’Arpino, C., Buch, S., Srivastava, S., Tchapmi, L. P., Tchapmi, M. E., Vainio, K., Wong, J., Fei-Fei, L., and Savarese, S. (2021) · 2012
Earlier work this paper cites.
Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillation.
Original
Zhao, C., and Hospedales, T. (2020) · 2012
Earlier work this paper cites.
The Arcade Learning Environment: An Evaluation Platform for General Agents.
Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M. (2013) · 2013
Earlier work this paper cites.
Near-optimal reinforcement learning in factored mdps
Osband, I., and Roy, B. V. (2014) · 2014
Earlier work this paper cites.
Contextual Markov Decision Processes.
Original
Hallak, A., Di Castro, D., and Mannor, S. (2015) · 2015
Earlier work this paper cites.
Deep learning and the information bottleneck principle
Tishby, N., and Zaslavsky, N. (2015) · 2015
Earlier work this paper cites.
OpenAI Gym.
Original
Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016) · 2016
Earlier work this paper cites.
Hidden parameter markov decision processes: A semiparametric regression approach for discovering latent task parametrizations
Doshi-Velez, F., and Konidaris, G. D. (2016) · 2016
Earlier work this paper cites.
Rl $2̂$: Fast reinforcement learning via slow reinforcement learning.
Original
Duan, Y., Schulman, J., Chen, X., Bartlett, P. L., Sutskever, I., and Abbeel, P. (2016) · 2016
Earlier work this paper cites.
The malmo platform for artificial intelligence experimentation
Johnson, M., Hofmann, K., Hutton, T., and Bignell, D. (2016) · 2016
Earlier work this paper cites.
ViZDoom: A Doom-based AI research platform for visual reinforcement learning
Kempka, M., Wydmuch, M., Runc, G., Toczek, J., and Jaśkowski, W. (2016) · 2016
Earlier work this paper cites.
OpenAI Gym: The DoomTakeCover-v0 environment
OpenAI, O. (2016) · 2016
Earlier work this paper cites.
Improved Regularization of Convolutional Neural Networks with Cutout.
Original
DeVries, T., and Taylor, G. W. (2017) · 2017
Earlier work this paper cites.
CARLA: An Open Urban Driving Simulator.
Original
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017) · 2017
Earlier work this paper cites.
Model-agnostic meta-learning for fast adaptation of deep networks
Finn, C., Abbeel, P., and Levine, S. (2017) · 2017
Earlier work this paper cites.
ADAPT: Zero-Shot Adaptive Policy Transfer for Stochastic Dynamical Systems.
Original
Harrison, J., Garg, A., Ivanovic, B., Zhu, Y., Savarese, S., Fei-Fei, L., and Pavone, M. (2017) · 2017
Earlier work this paper cites.
beta-vae: Learning basic visual concepts with a constrained variational framework
Higgins, I., Matthey, L., Pal, A., Burgess, C., Glorot, X., Botvinick, M., Mohamed, S., and Lerchner, A. (2017a) · 2017
Earlier work this paper cites.
DARLA: improving zero-shot transfer in reinforcement learning
Higgins, I., Pal, A., Rusu, A. A., Matthey, L., Burgess, C., Pritzel, A., Botvinick, M., Blundell, C., and Lerchner, A. (2017b) · 2017
Earlier work this paper cites.
Schema networks: Zero-shot transfer with a generative causal model of intuitive physics
Kansky, K., Silver, T., Mély, D. A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., Dorfman, N., Sidor, S., Phoenix, D. S., and George, D. (2017) · 2017
Earlier work this paper cites.

Similar

Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents.
Original
Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., and Bowling, M. (2017) · 2017
Cited alongside, same era.
Robust adversarial reinforcement learning
Pinto, L., Davidson, J., Sukthankar, R., and Gupta, A. (2017) · 2017
Cited alongside, same era.
Imagination-augmented agents for deep reinforcement learning
Racanière, S., Weber, T., Reichert, D. P., Buesing, L., Guez, A., Rezende, D. J., Badia, A. P., Vinyals, O., Heess, N., Li, Y., Pascanu, R., Battaglia, P. W., Hassabis, D., Silver, D., and Wierstra, D. (2017) · 2017
Cited alongside, same era.
CAD2RL: Real Single-Image Flight without a Single Real Image.
Original
Sadeghi, F., and Levine, S. (2017) · 2017
Cited alongside, same era.
Proximal Policy Optimization Algorithms.
Original
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017) · 2017
Cited alongside, same era.
Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World.
Original
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017) · 2017
Cited alongside, same era.
Learning to reinforcement learn.
Original
Wang, J. X., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J. Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M. (2017) · 2017
Cited alongside, same era.
Preparing for the Unknown: Learning a Universal Policy with Online System Identification.
Original
Yu, W., Tan, J., Liu, C. K., and Turk, G. (2017) · 2017
Cited alongside, same era.
Maximum a posteriori policy optimisation
Abdolmaleki, A., Springenberg, J. T., Tassa, Y., Munos, R., Heess, N., and Riedmiller, M. A. (2018) · 2018
Cited alongside, same era.
Autonomous agents modelling other agents: A comprehensive survey and open problems.
Albrecht, S. V., and Stone, P. (2018) · 2018
Cited alongside, same era.
Relational inductive biases, deep learning, and graph networks.
Original
Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gulcehre, C., Song, F., Ballard, A., Gilmer, J., Dahl, G., Vaswani, A., Allen, K., Nash, C., Langston, V., Dyer, C., Heess, N., Wierstra, D., Kohli, P., Botvinick, M., Vinyals, O., Li, Y., and Pascanu, R. (2018) · 2018
Cited alongside, same era.
TextWorld: A Learning Environment for Text-based Games.
Original
Côté, M.-A., Kádár, Á., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Tao, R. Y., Hausknecht, M., Asri, L. E., Adada, M., Tay, W., and Trischler, A. (2019) · 2018
Cited alongside, same era.

Then

Increasing generality in machine learning through procedural content generation.
Risi, S., and Togelius, J. (2020) · 2020
Later among the works it cites.
Mastering Atari, Go, chess and shogi by planning with a learned model.
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., Lillicrap, T., and Silver, D. (2020) · 2020
Later among the works it cites.
Observational overfitting in reinforcement learning
Song, X., Jiang, Y., Tu, S., Du, Y., and Neyshabur, B. (2020) · 2020
Later among the works it cites.
Learning Invariances for Policy Generalization.
Original
Tachet, R., Bachman, P., and van Seijen, H. (2020) · 2020
Later among the works it cites.
Neuroevolution of Self-Interpretable Agents.
Tang, Y., Nguyen, D., and Ha, D. (2020) · 2020
Later among the works it cites.
A survey of multi-task deep reinforcement learning.
Vithayathil Varghese, N., and Mahmoud, Q. H. (2020) · 2020
Later among the works it cites.
Improving Generalization in Reinforcement Learning with Mixture Regularization.
Original
Wang, K., Kang, B., Shao, J., and Feng, J. (2020) · 2020
Later among the works it cites.
Enhanced POET: open-ended reinforcement learning through unbounded invention of learning challenges and their solutions
Wang, R., Lehman, J., Rawal, A., Zhi, J., Li, Y., Clune, J., and Stanley, K. O. (2020) · 2020
Later among the works it cites.
Invariant causal prediction for block mdps
Zhang, A., Lyle, C., Sodhani, S., Filos, A., Kwiatkowska, M., Pineau, J., Gal, Y., and Precup, D. (2020) · 2020
Later among the works it cites.
Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: A Survey
Zhao, W., Queralta, J. P., and Westerlund, T. (2020) · 2020
Later among the works it cites.
RTFM: Generalising to Novel Environment Dynamics via Reading.
Original
Zhong, V., Rocktäschel, T., and Grefenstette, E. (2021) · 2020
Later among the works it cites.
Varibad: A very good method for bayes-adaptive deep RL via meta-learning
Zintgraf, L. M., Shiarlis, K., Igl, M., Schulze, S., Gal, Y., Hofmann, K., and Whiteson, S. (2020) · 2020
Later among the works it cites.
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning.
Original
Agarwal, R., Machado, M. C., Castro, P. S., and Bellemare, M. G. (2021) · 2021
Later among the works it cites.
A Survey of Exploration Methods in Reinforcement Learning.
Original
Amin, S., Gomrokchi, M., Satija, H., van Hoof, H., and Precup, D. (2021) · 2021
Later among the works it cites.
Procedural Generalization by Planning with Self-Supervised World Models.
Original
Anand, A., Walker, J., Li, Y., Vértes, E., Schrittwieser, J., Ozair, S., Weber, T., and Hamrick, J. B. (2021) · 2021
Later among the works it cites.
Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environment.
Original
Ball, P. J., Lu, C., Parker-Holder, J., and Roberts, S. (2021) · 2021
Later among the works it cites.
CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning.
Benjamins, C., Eimer, T., Schubert, F., Biedenkapp, A., Rosenhahn, B., Hutter, F., and Lindauer, M. (2021) · 2021
Later among the works it cites.
Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videos.
Original
Chen, A. S., Nair, S., and Finn, C. (2021) · 2021
Later among the works it cites.
Minimalistic Gridworld Environment (MiniGrid).
Chevalier-Boisvert, M. (2021) · 2021
Later among the works it cites.
DACBench: A Benchmark Library for Dynamic Algorithm Configuration.
Original
Eimer, T., Biedenkapp, A., Reimer, M., Adriaensen, S., Hutter, F., and Lindauer, M. (2021) · 2021
Later among the works it cites.
Robust Predictable Control.
Original
Eysenbach, B., Salakhutdinov, R., and Levine, S. (2021) · 2021
Later among the works it cites.
DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck.
Original
Fan, J., and Li, W. (2021) · 2021
Later among the works it cites.
SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policies.
Original
Fan, L., Wang, G., Huang, D.-A., Yu, Z., Fei-Fei, L., Zhu, Y., and Anandkumar, A. (2021) · 2021
Later among the works it cites.
Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability.
Original
Ghosh, D., Rahme, J., Kumar, A., Zhang, A., Adams, R. P., and Levine, S. (2021) · 2021
Later among the works it cites.
NovelGridworlds: A benchmark environment for detecting and adapting to novelties in open worlds.
Goel, S., Tatiya, G., Scheutz, M., and Sinapov, J. (2021) · 2021
Later among the works it cites.
Benchmarking the Spectrum of Agent Capabilities.
Original
Hafner, D. (2021) · 2021
Later among the works it cites.
A New Open-Source Off-Road Environment for Benchmark Generalization of Autonomous Driving.
Han, I., Park, D.-H., and Kim, K.-J. (2021) · 2021
Later among the works it cites.
Self-Supervised Policy Adaptation during Deployment.
Original
Hansen, N., Jangir, R., Sun, Y., Alenyà, G., Abbeel, P., Efros, A. A., Pinto, L., and Wang, X. (2021a) · 2021
Later among the works it cites.
Generalization in Reinforcement Learning by Soft Data Augmentation.
Original
Hansen, N., and Wang, X. (2021) · 2021
Later among the works it cites.
Online sparse reinforcement learning
Hao, B., Lattimore, T., Szepesvári, C., and Wang, M. (2021) · 2021
Later among the works it cites.
Off-Belief Learning.
Original
Hu, H., Lerer, A., Cui, B., Wu, D., Pineda, L., Brown, N., and Foerster, J. (2021) · 2021
Later among the works it cites.
AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learning.
Original
Huang, B., Feng, F., Lu, C., Magliacane, S., and Zhang, K. (2021) · 2021
Later among the works it cites.
Replay-Guided Adversarial Environment Design.
Original
Jiang, M., Dennis, M., Parker-Holder, J., Foerster, J., Grefenstette, E., and Rocktäschel, T. (2021a) · 2021
Later among the works it cites.
Parametric Generalization for Benchmarking Reinforcement Learning Algorithms.
Ke, N. R., Dasgupta, I., Chiappa, S., Goyal, A., Weber, T., Mitrovic, J., Hill, F., Chan, S. C. Y., Mozer, M. C., Rezende, D. J., and Kohli, P. (2021) · 2021
Later among the works it cites.
Towards Robust Bisimulation Metric Learning.
Original
Kemertas, M., and Aumentado-Armstrong, T. (2021) · 2021
Later among the works it cites.
Time Matters in Using Data Augmentation for Vision-based Deep Reinforcement Learning.
Original
Ko, B., and Ok, J. (2021) · 2021
Later among the works it cites.
MarsExplorer: Exploration of Unknown Terrains via Deep Reinforcement Learning and Procedurally Generated Environments.
Original
Koutras, D. I., Kapoutsis, A. C., Amanatiadis, A. A., and Kosmatopoulos, E. B. (2021) · 2021
Later among the works it cites.
RMA: Rapid Motor Adaptation for Legged Robots.
Original
Kumar, A., Fu, Z., Pathak, D., and Malik, J. (2021) · 2021
Later among the works it cites.
Language Conditioned Imitation Learning over Unstructured Data.
Original
Lynch, C., and Sermanet, P. (2021) · 2021
Later among the works it cites.
When Is Generalizable Reinforcement Learning Tractable?.
Original
Malik, D., Li, Y., and Ravikumar, P. (2021) · 2021
Later among the works it cites.
Cross-Trajectory Representation Learning for Zero-Shot Generalization in RL.
Original
Mazoure, B., Ahmed, A. M., MacAlpine, P., Hjelm, R. D., and Kolobov, A. (2021) · 2021
Later among the works it cites.
Procedural Content Generation: Better Benchmarks for Transfer Reinforcement Learning.
Original
Müller-Brockhausen, M., Preuss, M., and Plaat, A. (2021) · 2021
Later among the works it cites.
Recurrent Model-Free RL is a Strong Baseline for Many POMDPs.
Original
Ni, T., Eysenbach, B., and Salakhutdinov, R. (2021) · 2021
Later among the works it cites.
CORA: Benchmarks, Baselines, and Metrics as a Platform for Continual Reinforcement Learning Agents.
Original
Powers, S., Xing, E., Kolve, E., Mottaghi, R., and Gupta, A. (2021) · 2021
Later among the works it cites.
Decoupling Value and Policy for Generalization in Reinforcement Learning.
Original
Raileanu, R., and Fergus, R. (2021) · 2021
Later among the works it cites.
MDP Playground: A Design and Debug Testbed for Reinforcement Learning.
Original
Rajan, R., Diaz, J. L. B., Guttikonda, S., Ferreira, F., Biedenkapp, A., von Hartz, J. O., and Hutter, F. (2021) · 2021
Later among the works it cites.
MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research
Samvelyan, M., Kirk, R., Kurin, V., Parker-Holder, J., Jiang, M., Hambro, E., Petroni, F., Kuttler, H., Grefenstette, E., and Rocktäschel, T. (2021) · 2021
Later among the works it cites.
Online and Offline Reinforcement Learning by Planning with a Learned Model.
Original
Schrittwieser, J., Hubert, T., Mandhane, A., Barekatain, M., Antonoglou, I., and Silver, D. (2021) · 2021
Later among the works it cites.
Sparse Attention Guided Dynamic Value Estimation for Single-Task Multi-Scene Reinforcement Learning.
Original
Singh, J., and Zheng, L. (2021) · 2021
Later among the works it cites.
The Distracting Control Suite – A Challenging Benchmark for Reinforcement Learning from Pixels.
Original
Stone, A., Ramirez, O., Konolige, K., and Jonschkowski, R. (2021) · 2021
Later among the works it cites.
The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learning.
Original
Tang, Y., and Ha, D. (2021) · 2021
Later among the works it cites.
Open-Ended Learning Leads to Generally Capable Agents.
Original
Team, O. E. L., Stooke, A., Mahajan, A., Barros, C., Deck, C., Bauer, J., Sygnowski, J., Trebacz, M., Jaderberg, M., Mathieu, M., McAleese, N., Bradley-Schmieg, N., Wong, N., Porcel, N., Raileanu, R., Hughes-Fitt, S., Dalibard, V., and Czarnecki, W. M. (2021) · 2021
Later among the works it cites.
Neuro-algorithmic Policies enable Fast Combinatorial Generalization.
Original
Vlastelica, M., Rolínek, M., and Martius, G. (2021) · 2021
Later among the works it cites.
Alchemy: A structured task distribution for meta-reinforcement learning.
Original
Wang, J. X., King, M., Porcel, N., Kurth-Nelson, Z., Zhu, T., Deck, C., Choy, P., Cassin, M., Reynolds, M., Song, F., Buttimore, G., Reichert, D. P., Rabinowitz, N., Matthey, L., Hassabis, D., Lerchner, A., and Botvinick, M. (2021) · 2021
Later among the works it cites.
Unsupervised Visual Attention and Invariance for Reinforcement Learning.
Original
Wang, X., Lian, L., and Yu, S. X. (2021) · 2021
Later among the works it cites.
Dropout’s Dream Land: Generalization from Learned Simulators to Reality
Wellmer, Z., and Kwok, J. T. (2021) · 2021
Later among the works it cites.
HALMA: Humanlike Abstraction Learning Meets Affordance in Rapid Problem Solving.
Original
Xie, S., Ma, X., Yu, P., Zhu, Y., Wu, Y. N., and Zhu, S.-C. (2021) · 2021
Later among the works it cites.
KitchenShift: Evaluating Zero-Shot Generalization of Imitation-Based Policy Learning Under Domain Shifts
Xing, E., Gupta, A., Powers, S., and Dean, V. (2021b) · 2021
Later among the works it cites.
Phy-Q: A Benchmark for Physical Reasoning.
Original
Xue, C., Pinto, V., Gamage, C., Nikonova, E., Zhang, P., and Renz, J. (2021) · 2021
Later among the works it cites.
Learning Invariant Representations for Reinforcement Learning without Reconstruction.
Original
Zhang, A., McAllister, R., Calandra, R., Gal, Y., and Levine, S. (2021) · 2021
Later among the works it cites.
Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentation.
Original
Zhang, H., and Guo, Y. (2021) · 2021
Later among the works it cites.
Domain Generalization with MixStyle.
Original
Zhou, K., Yang, Y., Qiao, Y., and Xiang, T. (2021) · 2021
Later among the works it cites.
Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning.
Original
Zintgraf, L., Feng, L., Lu, C., Igl, M., Hartikainen, K., Hofmann, K., and Whiteson, S. (2021) · 2021
Later among the works it cites.
A Survey of Explainable Reinforcement Learning.
Milani, S., Topin, N., Veloso, M., and Fang, F. (2022) · 2022
Later among the works it cites.
Block Contextual MDPs for Continual Learning
Sodhani, S., Meier, F., Pineau, J., and Zhang, A. (2022) · 2022
Later among the works it cites.
Domain Generalization: A Survey.
Zhou, K., Liu, Z., Qiao, Y., Xiang, T., and Loy, C. C. (2022) · 2022
Later among the works it cites.
Leveraging procedural generation to benchmark reinforcement learning
Cobbe, K., Hesse, C., Hilton, J., and Schulman, J. (2020a) · 2056
Later among the works it cites.
Transfer learning for related reinforcement learning tasks via image-to-image translation
Gamrian, S., and Goldberg, Y. (2019) · 2072
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…