Benchmarking Bonus-Based Exploration Methods on the Arcade Learning Environment

Pixel recurrent neural networks

Original

Van den Oord, A., Kalchbrenner, N., and Kavukcuoglu, K · 1908

Earlier work this paper cites.

An analysis of model-based interval estimation for Markov decision processes

Strehl, A. L. and Littman, M. L · 2008

Earlier work this paper cites.

The arcade learning environment: An evaluation platform for general agents

Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M · 2013

Earlier work this paper cites.

Skip context tree switching

Bellemare, M., Veness, J., and Talvitie, E · 2014

Earlier work this paper cites.

Human-level control through deep reinforcement learning

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al · 2015

Earlier work this paper cites.

Prioritized experience replay

Original

Schaul, T., Quan, J., Antonoglou, I., and Silver, D · 2015

Earlier work this paper cites.

Incentivizing exploration in reinforcement learning with deep predictive models

Original

Stadie, B. C., Levine, S., and Abbeel, P · 2015

Earlier work this paper cites.

Unifying count-based exploration and intrinsic motivation

Bellemare, M., Srinivasan, S., Ostrovski, G., Schaul, T., Saxton, D., and Munos, R · 2016

Earlier work this paper cites.

Asynchronous methods for deep reinforcement learning

Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K · 2016

Earlier work this paper cites.

Deep exploration via bootstrapped dqn

Osband, I., Blundell, C., Pritzel, A., and Van Roy, B · 2016

Cited alongside, same era.

A distributional perspective on reinforcement learning

Bellemare, M. G., Dabney, W., and Munos, R · 2017

Cited alongside, same era.

The uncertainty bellman equation and exploration

Original

O’Donoghue, B., Osband, I., Munos, R., and Mnih, V · 2017

Cited alongside, same era.

Count-based exploration with Neural Density Models

Ostrovski, G., Bellemare, M. G., van den Oord, A., and Munos, R · 2017

Cited alongside, same era.

Curiosity-driven exploration by self-supervised prediction

Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T · 2017

Cited alongside, same era.

# Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning

Tang, H., Houthooft, R., Foote, D., Stooke, A., Chen, O. X., Duan, Y., Schulman, J., DeTurck, F., and Abbeel, P · 2017

Dora the explorer: Directed outreaching reinforcement action-selection

Original

Choshen, L., Fox, L., and Loewenstein, Y · 2018

Later among the works it cites.

Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures

Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., Legg, S., and Kavukcuoglu, K · 2018

Later among the works it cites.

Noisy networks for exploration

Fortunato, M., Azar, M. G., Piot, B., Menick, J., Osband, I., Graves, A., Mnih, V., Munos, R., Hassabis, D., Pietquin, O., et al · 2018

Later among the works it cites.

Deep reinforcement learning that matters

Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D · 2018

Later among the works it cites.

Rainbow: Combining improvements in deep reinforcement learning

Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D · 2018

Later among the works it cites.