2019

Go-Explore: a New Approach for Hard-Exploration Problems

Ecoffet, Adrien, Huizinga, Joost, Lehman, Joel et al.

Understand

A grand challenge in reinforcement learning is intelligent exploration, especially when rewards are sparse or deceptive.

Two Atari games serve as benchmarks for such hard-exploration domains: Montezuma's Revenge and Pitfall.
On both games, current RL algorithms perform poorly, even those with intrinsic motivation, which is the dominant method to improve performance on hard-exploration domains.
To address this shortfall, we introduce a new algorithm called Go-Explore.

Built on

Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 1937
Earlier work this paper cites.
Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 1937
Earlier work this paper cites.
Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 1937
Earlier work this paper cites.
Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 1937
Earlier work this paper cites.
A note on two problems in connexion with graphs
E. W. Dijkstra · 1959
Earlier work this paper cites.
A note on two problems in connexion with graphs
E. W. Dijkstra · 1959
Earlier work this paper cites.
A formal basis for the heuristic determination of minimum cost paths
P. E. Hart, N. J. Nilsson, and B. Raphael · 1968
Earlier work this paper cites.
A formal basis for the heuristic determination of minimum cost paths
P. E. Hart, N. J. Nilsson, and B. Raphael · 1968
Earlier work this paper cites.
Genetic Algorithms in Search, Optimization and Machine Learning
David E. Goldberg · 1989
Earlier work this paper cites.
Genetic Algorithms in Search, Optimization and Machine Learning
David E. Goldberg · 1989
Earlier work this paper cites.
Curious model-building control systems
Jürgen Schmidhuber · 1991
Earlier work this paper cites.
Curious model-building control systems
Jürgen Schmidhuber · 1991
Earlier work this paper cites.
Reinforcement learning: An introduction , volume 1
Richard S Sutton and Andrew G Barto · 1998
Earlier work this paper cites.
Rapidly-exploring random trees: A new tool for path planning
Steven M. Lavalle · 1998
Earlier work this paper cites.
Reinforcement learning: An introduction , volume 1
Richard S Sutton and Andrew G Barto · 1998
Earlier work this paper cites.
Rapidly-exploring random trees: A new tool for path planning
Steven M. Lavalle · 1998
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
Robert M French · 1999
Earlier work this paper cites.
Catastrophic forgetting in connectionist networks
Robert M French · 1999
Earlier work this paper cites.
Near-optimal reinforcement learning in polynomial time
Michael Kearns and Satinder Singh · 2002
Earlier work this paper cites.
Near-optimal reinforcement learning in polynomial time
Michael Kearns and Satinder Singh · 2002
Earlier work this paper cites.
Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts
Jürgen Schmidhuber · 2006
Earlier work this paper cites.
Bandit based monte-carlo planning
Levente Kocsis and Csaba Szepesvári · 2006
Earlier work this paper cites.
Improved monte-carlo search
Levente Kocsis, Csaba Szepesvári, and Jan Willemson · 2006
Earlier work this paper cites.
Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts
Jürgen Schmidhuber · 2006
Earlier work this paper cites.
Bandit based monte-carlo planning
Levente Kocsis and Csaba Szepesvári · 2006
Earlier work this paper cites.
Improved monte-carlo search
Levente Kocsis, Csaba Szepesvári, and Jan Willemson · 2006
Earlier work this paper cites.
Bootstrap methods and applications
A. M. Zoubir and D. Robert Iskander · 2007
Earlier work this paper cites.
Bootstrap methods and applications
A. M. Zoubir and D. Robert Iskander · 2007
Earlier work this paper cites.
State-dependent exploration for policy gradient methods
Thomas Rückstieß, Martin Felder, and Jürgen Schmidhuber · 2008
Earlier work this paper cites.
Monte-carlo tree search: A new framework for game ai
Guillaume Chaslot, Sander Bakkes, István Szita, and Pieter Spronck · 2008
Earlier work this paper cites.
State-dependent exploration for policy gradient methods
Thomas Rückstieß, Martin Felder, and Jürgen Schmidhuber · 2008
Earlier work this paper cites.
Monte-carlo tree search: A new framework for game ai
Guillaume Chaslot, Sander Bakkes, István Szita, and Pieter Spronck · 2008
Earlier work this paper cites.
What is intrinsic motivation? a typology of computational approaches
Pierre-Yves Oudeyer and Frederic Kaplan · 2009
Earlier work this paper cites.
What is intrinsic motivation? a typology of computational approaches
Pierre-Yves Oudeyer and Frederic Kaplan · 2009
Earlier work this paper cites.
Deep auto-encoder neural networks in reinforcement learning
Sascha Lange and Martin Riedmiller · 2010
Earlier work this paper cites.
Deep auto-encoder neural networks in reinforcement learning
Sascha Lange and Martin Riedmiller · 2010
Earlier work this paper cites.
Novelty search and the problem with objectives
Joel Lehman and Kenneth O. Stanley · 2011
Earlier work this paper cites.
Roulette-wheel selection via stochastic acceptance
Original
Adam Lipowski and Dorota Lipowska · 2011
Earlier work this paper cites.
Novelty search and the problem with objectives
Joel Lehman and Kenneth O. Stanley · 2011
Earlier work this paper cites.
Roulette-wheel selection via stochastic acceptance
Original
Adam Lipowski and Dorota Lipowska · 2011
Earlier work this paper cites.
A survey of monte carlo tree search methods
Cameron Browne, Edward Jack Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez Liebana, Spyridon Samothrakis, and Simon Colton · 2012
Earlier work this paper cites.
Investigating contingency awareness using atari 2600 games
Marc G. Bellemare, Joel Veness, and Michael H. Bowling · 2012
Earlier work this paper cites.
A survey of monte carlo tree search methods
Cameron Browne, Edward Jack Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez Liebana, Spyridon Samothrakis, and Simon Colton · 2012
Earlier work this paper cites.
Investigating contingency awareness using atari 2600 games
Marc G. Bellemare, Joel Veness, and Michael H. Bowling · 2012
Earlier work this paper cites.
Intrinsic motivation and reinforcement learning
Andrew G Barto · 2013
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling · 2013
Earlier work this paper cites.
The transferability approach: Crossing the reality gap in evolutionary robotics
Sylvain Koos, Jean-Baptiste Mouret, and Stéphane Doncieux · 2013
Earlier work this paper cites.
Evolving gaits for physical robots with the hyperneat generative encoding: the benefits of simulation
S. Lee, J. Yosinski, K. Glette, H. Lipson, and J. Clune · 2013
Earlier work this paper cites.
Intrinsic motivation and reinforcement learning
Andrew G Barto · 2013
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling · 2013
Earlier work this paper cites.
The transferability approach: Crossing the reality gap in evolutionary robotics
Sylvain Koos, Jean-Baptiste Mouret, and Stéphane Doncieux · 2013
Earlier work this paper cites.
Evolving gaits for physical robots with the hyperneat generative encoding: the benefits of simulation
S. Lee, J. Yosinski, K. Glette, H. Lipson, and J. Clune · 2013
Earlier work this paper cites.
Deep learning for real-time atari game play using offline monte-carlo tree search planning
Xiaoxiao Guo, Satinder P. Singh, Honglak Lee, Richard L. Lewis, and Xiaoshi Wang · 2014
Earlier work this paper cites.
Deep learning for real-time atari game play using offline monte-carlo tree search planning
Xiaoxiao Guo, Satinder P. Singh, Honglak Lee, Richard L. Lewis, and Xiaoshi Wang · 2014
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis · 2015
Earlier work this paper cites.
Neural modularity helps organisms evolve to learn new skills without forgetting old skills
Kai Olav Ellefsen, Jean-Baptiste Mouret, Jeff Clune, and Josh C Bongard · 2015
Earlier work this paper cites.
Classical planning with simulators: Results on the atari video games
Nir Lipovetzky, Miquel Ramírez, and Hector Geffner · 2015
Earlier work this paper cites.
Robots that can adapt like animals
A. Cully, J. Clune, D. Tarapore, and J.-B. Mouret · 2015
Earlier work this paper cites.
Massively parallel methods for deep reinforcement learning
Original
Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al · 2015
Earlier work this paper cites.
Illuminating search spaces by mapping elites
Original
Jean-Baptiste Mouret and Jeff Clune · 2015
Earlier work this paper cites.
A comprehensive survey on safe reinforcement learning
Javier Garcıa and Fernando Fernández · 2015
Earlier work this paper cites.
Continuous control with deep reinforcement learning
Original
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra · 2015
Earlier work this paper cites.
Incentivizing exploration in reinforcement learning with deep predictive models
Original
Bradly C Stadie, Sergey Levine, and Pieter Abbeel · 2015
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis · 2015
Earlier work this paper cites.

Similar

Neural modularity helps organisms evolve to learn new skills without forgetting old skills
Kai Olav Ellefsen, Jean-Baptiste Mouret, Jeff Clune, and Josh C Bongard · 2015
Cited alongside, same era.
Classical planning with simulators: Results on the atari video games
Nir Lipovetzky, Miquel Ramírez, and Hector Geffner · 2015
Cited alongside, same era.
Robots that can adapt like animals
A. Cully, J. Clune, D. Tarapore, and J.-B. Mouret · 2015
Cited alongside, same era.
Massively parallel methods for deep reinforcement learning
Original
Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al · 2015
Cited alongside, same era.
Illuminating search spaces by mapping elites
Original
Jean-Baptiste Mouret and Jeff Clune · 2015
Cited alongside, same era.
A comprehensive survey on safe reinforcement learning
Javier Garcıa and Fernando Fernández · 2015
Cited alongside, same era.
Continuous control with deep reinforcement learning
Original
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra · 2015
Cited alongside, same era.
Incentivizing exploration in reinforcement learning with deep predictive models
Original
Bradly C Stadie, Sergey Levine, and Pieter Abbeel · 2015
Cited alongside, same era.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016
Cited alongside, same era.
Unifying count-based exploration and intrinsic motivation
Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos · 2016
Cited alongside, same era.
Concrete problems in ai safety
Original
Dario Amodei, Chris Olah, Jacob Steinhardt, Paul F. Christiano, John Schulman, and Dan Mané · 2016
Cited alongside, same era.
Generative adversarial imitation learning
Jonathan Ho and Stefano Ermon · 2016
Cited alongside, same era.

Then

Count-based exploration with neural density models
Georg Ostrovski, Marc G. Bellemare, Aäron van den Oord, and Rémi Munos · 2017
Later among the works it cites.
Uncertainty-aware reinforcement learning for collision avoidance
Original
Gregory Kahn, Adam Villaflor, Vitchyr Pong, Pieter Abbeel, and Sergey Levine · 2017
Later among the works it cites.
Building machines that learn and think like people
Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J Gershman · 2017
Later among the works it cites.
Hindsight experience replay
Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba · 2017
Later among the works it cites.
Proximal policy optimization algorithms
Original
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017
Later among the works it cites.
Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning
Original
Felipe Petroski Such, Vashisht Madhavan, Edoardo Conti, Joel Lehman, Kenneth O. Stanley, and Jeff Clune · 2017
Later among the works it cites.
Evolution strategies as a scalable alternative to reinforcement learning
Original
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever · 2017
Later among the works it cites.
A distributional perspective on reinforcement learning
Marc G. Bellemare, Will Dabney, and Rémi Munos · 2017
Later among the works it cites.
Domain randomization for transferring deep neural networks from simulation to the real world
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel · 2017
Later among the works it cites.
Intrinsically motivated goal exploration processes with automatic curriculum learning
Original
Sébastien Forestier, Yoan Mollard, and Pierre-Yves Oudeyer · 2017
Later among the works it cites.
Back to basics: Benchmarking canonical evolution strategies for playing atari
Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter · 2018
Later among the works it cites.
Exploration by random network distillation
Original
Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov · 2018
Later among the works it cites.
Deep q-learning from demonstrations
Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan, John Quan, Andrew Sendonaris, Ian Osband, Gabriel Dulac-Arnold, John Agapiou, Joel Z. Leibo, and Audrunas Gruslys · 2018
Later among the works it cites.
Observe and look further: Achieving consistent performance on atari
Original
Tobias Pohlen, Bilal Piot, Todd Hester, Mohammad Gheshlaghi Azar, Dan Horgan, David Budden, Gabriel Barth-Maron, Hado van Hasselt, John Quan, Mel Večerík, et al · 2018
Later among the works it cites.
Learning montezuma’s revenge from a single demonstration
Original
Tim Salimans and Richard Chen · 2018
Later among the works it cites.
Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents
Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew J. Hausknecht, and Michael H. Bowling · 2018
Later among the works it cites.
Deep curiosity search: Intra-life exploration improves performance on challenging deep reinforcement learning problems
Original
Christopher Stanton and Jeff Clune · 2018
Later among the works it cites.
The uncertainty bellman equation and exploration
Brendan O’Donoghue, Ian Osband, Rémi Munos, and Volodymyr Mnih · 2018
Later among the works it cites.
Contingency-aware exploration in reinforcement learning
Original
Jongwook Choi, Yijie Guo, Marcin Moczulski, Junhyuk Oh, Neal Wu, Mohammad Norouzi, and Honglak Lee · 2018
Later among the works it cites.
Distributed prioritized experience replay
Original
Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado van Hasselt, and David Silver · 2018
Later among the works it cites.
Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures
Lasse Espeholt, Hubert Soyer, Rémi Munos, Karen Simonyan, Volodymyr Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu · 2018
Later among the works it cites.
Playing hard exploration games by watching youtube
Original
Yusuf Aytar, Tobias Pfaff, David Budden, Tom Le Paine, Ziyu Wang, and Nando de Freitas · 2018
Later among the works it cites.
An atari model zoo for analyzing, visualizing, and comparing deep reinforcement learning agents
Original
Felipe Petroski Such, Vashisht Madhavan, Rosanne Liu, Rui Wang, Pablo Samuel Castro, Yulun Li, Ludwig Schubert, Marc Bellemare, Jeff Clune, and Joel Lehman · 2018
Later among the works it cites.
Montezuma’s revenge solved by go-explore, a new algorithm for hard-exploration problems (sets records on pitfall, too)
Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O Stanley, and Jeff Clune · 2018
Later among the works it cites.
Robustness to out-of-distribution inputs via task-aware generative uncertainty
Original
Rowan McAllister, Gregory Kahn, Jeff Clune, and Sergey Levine · 2018
Later among the works it cites.
Learning dexterous in-hand manipulation
Original
Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al · 2018
Later among the works it cites.
Backplay: "man muss immer umkehren"
Original
Cinjon Resnick, Roberta Raileanu, Sanyam Kapoor, Alex Peysakhovich, Kyunghyun Cho, and Joan Bruna · 2018
Later among the works it cites.
Rainbow: Combining improvements in deep reinforcement learning
Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Gheshlaghi Azar, and David Silver · 2018
Later among the works it cites.
Trial without error: Towards safe reinforcement learning via human intervention
William Saunders, Girish Sastry, Andreas Stuhlmueller, and Owain Evans · 2018
Later among the works it cites.
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms
Cédric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer · 2018
Later among the works it cites.
Self-imitation learning
Junhyuk Oh, Yijie Guo, Satinder Singh, and Honglak Lee · 2018
Later among the works it cites.
Recall traces: Backtracking models for efficient reinforcement learning
Original
Anirudh Goyal, Philemon Brakel, William Fedus, Timothy P. Lillicrap, Sergey Levine, Hugo Larochelle, and Yoshua Bengio · 2018
Later among the works it cites.
Fractal ai: A fragile theory of intelligence
Original
Sergio Hernandez Cerezo and Guillem Duran Ballester · 2018
Later among the works it cites.
Taking the scenic route: Automatic exploration for videogames
Original
Zeping Zhan, Batu Aytemiz, and Adam M Smith · 2018
Later among the works it cites.
Back to basics: Benchmarking canonical evolution strategies for playing atari
Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter · 2018
Later among the works it cites.
Exploration by random network distillation
Original
Yuri Burda, Harrison Edwards, Amos Storkey, and Oleg Klimov · 2018
Later among the works it cites.
Deep q-learning from demonstrations
Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Dan Horgan, John Quan, Andrew Sendonaris, Ian Osband, Gabriel Dulac-Arnold, John Agapiou, Joel Z. Leibo, and Audrunas Gruslys · 2018
Later among the works it cites.
Observe and look further: Achieving consistent performance on atari
Original
Tobias Pohlen, Bilal Piot, Todd Hester, Mohammad Gheshlaghi Azar, Dan Horgan, David Budden, Gabriel Barth-Maron, Hado van Hasselt, John Quan, Mel Večerík, et al · 2018
Later among the works it cites.
Learning montezuma’s revenge from a single demonstration
Original
Tim Salimans and Richard Chen · 2018
Later among the works it cites.
Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents
Marlos C. Machado, Marc G. Bellemare, Erik Talvitie, Joel Veness, Matthew J. Hausknecht, and Michael H. Bowling · 2018
Later among the works it cites.
Deep curiosity search: Intra-life exploration improves performance on challenging deep reinforcement learning problems
Original
Christopher Stanton and Jeff Clune · 2018
Later among the works it cites.
The uncertainty bellman equation and exploration
Brendan O’Donoghue, Ian Osband, Rémi Munos, and Volodymyr Mnih · 2018
Later among the works it cites.
Contingency-aware exploration in reinforcement learning
Original
Jongwook Choi, Yijie Guo, Marcin Moczulski, Junhyuk Oh, Neal Wu, Mohammad Norouzi, and Honglak Lee · 2018
Later among the works it cites.
Distributed prioritized experience replay
Original
Dan Horgan, John Quan, David Budden, Gabriel Barth-Maron, Matteo Hessel, Hado van Hasselt, and David Silver · 2018
Later among the works it cites.
Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures
Lasse Espeholt, Hubert Soyer, Rémi Munos, Karen Simonyan, Volodymyr Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, Shane Legg, and Koray Kavukcuoglu · 2018
Later among the works it cites.
Playing hard exploration games by watching youtube
Original
Yusuf Aytar, Tobias Pfaff, David Budden, Tom Le Paine, Ziyu Wang, and Nando de Freitas · 2018
Later among the works it cites.
An atari model zoo for analyzing, visualizing, and comparing deep reinforcement learning agents
Original
Felipe Petroski Such, Vashisht Madhavan, Rosanne Liu, Rui Wang, Pablo Samuel Castro, Yulun Li, Ludwig Schubert, Marc Bellemare, Jeff Clune, and Joel Lehman · 2018
Later among the works it cites.
Montezuma’s revenge solved by go-explore, a new algorithm for hard-exploration problems (sets records on pitfall, too)
Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O Stanley, and Jeff Clune · 2018
Later among the works it cites.
Robustness to out-of-distribution inputs via task-aware generative uncertainty
Original
Rowan McAllister, Gregory Kahn, Jeff Clune, and Sergey Levine · 2018
Later among the works it cites.
Learning dexterous in-hand manipulation
Original
Marcin Andrychowicz, Bowen Baker, Maciek Chociej, Rafal Jozefowicz, Bob McGrew, Jakub Pachocki, Arthur Petron, Matthias Plappert, Glenn Powell, Alex Ray, et al · 2018
Later among the works it cites.
Backplay: "man muss immer umkehren"
Original
Cinjon Resnick, Roberta Raileanu, Sanyam Kapoor, Alex Peysakhovich, Kyunghyun Cho, and Joan Bruna · 2018
Later among the works it cites.
Rainbow: Combining improvements in deep reinforcement learning
Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Gheshlaghi Azar, and David Silver · 2018
Later among the works it cites.
Trial without error: Towards safe reinforcement learning via human intervention
William Saunders, Girish Sastry, Andreas Stuhlmueller, and Owain Evans · 2018
Later among the works it cites.
GEP-PG: Decoupling Exploration and Exploitation in Deep Reinforcement Learning Algorithms
Cédric Colas, Olivier Sigaud, and Pierre-Yves Oudeyer · 2018
Later among the works it cites.
Self-imitation learning
Junhyuk Oh, Yijie Guo, Satinder Singh, and Honglak Lee · 2018
Later among the works it cites.
Recall traces: Backtracking models for efficient reinforcement learning
Original
Anirudh Goyal, Philemon Brakel, William Fedus, Timothy P. Lillicrap, Sergey Levine, Hugo Larochelle, and Yoshua Bengio · 2018
Later among the works it cites.
Fractal ai: A fragile theory of intelligence
Original
Sergio Hernandez Cerezo and Guillem Duran Ballester · 2018
Later among the works it cites.
Taking the scenic route: Automatic exploration for videogames
Original
Zeping Zhan, Batu Aytemiz, and Adam M Smith · 2018
Later among the works it cites.
Learning abstract models for long-horizon exploration, 2019
Evan Zheran Liu, Ramtin Keramati, Sudarshan Seshadri, Kelvin Guu, Panupong Pasupat, Emma Brunskill, and Percy Liang · 2019
Closest in time.
Fast exploration with simplified models and approximately optimistic planning in model based reinforcement learning, 2019
Ramtin Keramati, Jay Whang, Patrick Cho, and Emma Brunskill · 2019
Closest in time.
Explicit recall for efficient exploration, 2019
Honghua Dong, Jiayuan Mao, Xinyue Cui, and Lihong Li · 2019
Closest in time.
Learning abstract models for long-horizon exploration, 2019
Evan Zheran Liu, Ramtin Keramati, Sudarshan Seshadri, Kelvin Guu, Panupong Pasupat, Emma Brunskill, and Percy Liang · 2019
Closest in time.
Fast exploration with simplified models and approximately optimistic planning in model based reinforcement learning, 2019
Ramtin Keramati, Jay Whang, Patrick Cho, and Emma Brunskill · 2019
Closest in time.
Explicit recall for efficient exploration, 2019
Honghua Dong, Jiayuan Mao, Xinyue Cui, and Lihong Li · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…