2020

Rotation, Translation, and Cropping for Zero-Shot Generalization

Ye, Chang, Khalifa, Ahmed, Bontrager, Philip et al.

Understand

Deep Reinforcement Learning (DRL) has shown impressive performance on domains with visual inputs, in particular various games.

However, the agent is usually trained on a fixed environment, e.g.
a fixed number of levels.
A growing mass of evidence suggests that these trained models fail to generalize to even slight variations of the environments they were trained on.

Built on

C. J. Watkins and P. Dayan, “Q-learning,” Machine learning , vol. 8, no. 3-4, 1992
1992
Earlier work this paper cites.
C. Kwok and D. Fox, “Reinforcement learning for sensing strategies,” in International Conference on Intelligent Robots and Systems , vol. 4. IEEE, 2004
2004
Earlier work this paper cites.
D. E. Broadbent, Perception and communication . Elsevier, 2013
2013
Earlier work this paper cites.
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al. , “Human-level control through deep reinforcement learning,” Nature , vol. 518, no. 7540, 2015
2015
Earlier work this paper cites.
G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba, “Openai gym,” 2016
2016
Earlier work this paper cites.
V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, and K. Kavukcuoglu, “Asynchronous methods for deep reinforcement learning,” in International conference on machine learning , 2016
2016
Earlier work this paper cites.
P. Dhariwal, C. Hesse, O. Klimov, A. Nichol, M. Plappert, A. Radford, J. Schulman, S. Sidor, Y. Wu, and P. Zhokhov, “Openai baselines,” 2017
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems , 2017, pp. 5998–6008
2017
Earlier work this paper cites.
N. Justesen, R. R. Torrado, P. Bontrager, A. Khalifa, J. Togelius, and S. Risi, “Illuminating generalization in deep reinforcement learning through procedural level generation,” in Deep RL Workshop NeurIPS 2018 , 2018
2018
Earlier work this paper cites.
M. Wydmuch, M. Kempka, and W. Jaśkowski, “Vizdoom competitions: Playing doom from pixels,” Transactions on Games , vol. 11, no. 3, 2018
2018
Earlier work this paper cites.

Similar

R. R. Torrado, P. Bontrager, J. Togelius, J. Liu, and D. Perez-Liebana, “Deep reinforcement learning for general video game ai,” in Computational Intelligence and Games . IEEE, 2018
2018
Cited alongside, same era.
M. C. Machado, M. G. Bellemare, E. Talvitie, J. Veness, M. Hausknecht, and M. Bowling, “Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents,” Journal of Artificial Intelligence Research , vol. 61, 2018
2018
Cited alongside, same era.
A. Zhang, Y. Wu, and J. Pineau, “Natural environment benchmarks for reinforcement learning,” arXiv preprint arXiv:1811.06032 , 2018
Original
2018
Cited alongside, same era.
L. Espeholt, H. Soyer, R. Munos, K. Simonyan, V. Mnih, T. Ward, Y. Doron, V. Firoiu, T. Harley, I. Dunning et al. , “Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures,” in ICML , 2018
2018
Cited alongside, same era.
X. Wang, R. Girshick, A. Gupta, and K. He, “Non-local neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 7794–7803
2018
Cited alongside, same era.
D. Ha and J. Schmidhuber, “Recurrent world models facilitate policy evolution,” in Advances in Neural Information Processing Systems , 2018, pp. 2450–2462
2018
Cited alongside, same era.
O. Vinyals, I. Babuschkin, W. M. Czarnecki, M. Mathieu, A. Dudzik, J. Chung, D. H. Choi, R. Powell, T. Ewalds, P. Georgiev et al. , “Grandmaster level in starcraft ii using multi-agent reinforcement learning,” Nature , vol. 575, no. 7782, 2019
2019
Cited alongside, same era.
G. Cuccu, J. Togelius, and P. Cudré-Mauroux, “Playing atari with six neurons,” in Autonomous Agents and Multi-Agent Systems . IFAAMAS, 2019
2019
Cited alongside, same era.
K. Cobbe, O. Klimov, C. Hesse, T. Kim, and J. Schulman, “Quantifying generalization in reinforcement learning,” in International Conference on Machine Learning , 2019, pp. 1282–1289
2019
Cited alongside, same era.
N. Justesen, P. Bontrager, J. Togelius, and S. Risi, “Deep learning for video game playing,” Transactions on Games , 2019
2019
Cited alongside, same era.

Then

A. Juliani, A. Khalifa, V.-P. Berges, J. Harper, E. Teng, H. Henry, A. Crespi, J. Togelius, and D. Lange, “Obstacle tower: a generalization challenge in vision, control, and planning,” in Proceedings of the 28th International Joint Conference on Artificial Intelligence . AAAI Press, 2019, pp. 2684–2691
2019
Later among the works it cites.
L. Harries, S. Lee, J. Rzepecki, K. Hofmann, and S. Devlin, “Mazeexplorer: A customisable 3d benchmark for assessing generalisation in reinforcement learning,” in Conference on Games . IEEE, 2019
2019
Later among the works it cites.
M. Igl, K. Ciosek, Y. Li, S. Tschiatschek, C. Zhang, S. Devlin, and K. Hofmann, “Generalization in reinforcement learning with selective noise injection and information bottleneck,” in Advances in Neural Information Processing Systems , 2019
2019
Later among the works it cites.
K. Cobbe, C. Hesse, J. Hilton, and J. Schulman, “Leveraging procedural generation to benchmark reinforcement learning,” arXiv preprint arXiv:1912.01588 , 2019
Original
2019
Later among the works it cites.
X. Song, Y. Jiang, S. Tu, Y. Du, and B. Neyshabur, “Observational overfitting in reinforcement learning,” in ICLR , 2020
2020
Closest in time.
B. Baker, I. Kanitscheider, T. Markov, Y. Wu, G. Powell, B. McGrew, and I. Mordatch, “Emergent tool use from multi-agent autocurricula,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=SkxpxJBKwS
2020
Closest in time.
M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srinivas, “Reinforcement learning with augmented data,” arXiv preprint arXiv:2004.14990 , 2020
Original
2020
Closest in time.
A. Srinivas, M. Laskin, and P. Abbeel, “Curl: Contrastive unsupervised representations for reinforcement learning,” arXiv preprint arXiv:2004.04136 , 2020
Original
2020
Closest in time.
I. Kostrikov, D. Yarats, and R. Fergus, “Image augmentation is all you need: Regularizing deep reinforcement learning from pixels,” arXiv preprint arXiv:2004.13649 , 2020
Original
2020
Closest in time.
K. Lee, K. Lee, J. Shin, and H. Lee, “Network randomization: A simple technique for generalization in deep reinforcement learning,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=HJgcvJBFvB
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…