2020

Model-based Reinforcement Learning: A Survey

Moerland, Thomas M., Broekens, Joost, Plaat, Aske et al.

Understand

Sequential decision making, commonly formalized as Markov Decision Process (MDP) optimization, is a important challenge in artificial intelligence.

Two key approaches to this problem are reinforcement learning (RL) and planning.
This paper presents a survey of the integration of both fields, better known as model-based reinforcement learning.
Model-based RL has two main steps.

Built on

Go-explore: a new approach for hard-exploration problems
Original
Ecoffet, A., Huizinga, J., Lehman, J., Stanley, K. O., and Clune, J. (2019) · 1901
Earlier work this paper cites.
Latent normalizing flows for discrete sequences
Original
Ziegler, Z. M. and Rush, A. M. (2019) · 1901
Earlier work this paper cites.
Learning Dynamics Model in Reinforcement Learning by Incorporating the Long Term Future
Original
Ke, N. R., Singh, A., Touati, A., Goyal, A., Bengio, Y., Parikh, D., and Batra, D. (2019) · 1903
Earlier work this paper cites.
Unsupervised discovery of parts, structure, and dynamics
Original
Xu, Z., Liu, Z., Sun, C., Murphy, K., Freeman, W. T., Tenenbaum, J. B., and Wu, J. (2019) · 1903
Earlier work this paper cites.
Policy Gradient Search: Online Planning and Expert Iteration without Search Trees
Original
Anthony, T., Nishihara, R., Moritz, P., Salimans, T., and Schulman, J. (2019) · 1904
Earlier work this paper cites.
A short survey on memory based reinforcement learning
Original
Ramani, D. (2019) · 1904
Earlier work this paper cites.
Tight Regret Bounds for Model-Based Reinforcement Learning with Greedy Policies
Original
Efroni, Y., Merlis, N., Ghavamzadeh, M., and Mannor, S. (2019b) · 1905
Earlier work this paper cites.
Cobra: Data-efficient model-based rl through unsupervised object discovery and curiosity-driven exploration
Original
Watters, N., Matthey, L., Bosnjak, M., Burgess, C. P., and Lerchner, A. (2019) · 1905
Earlier work this paper cites.
Benchmarking Model-Based Reinforcement Learning
Original
Wang, T., Bao, X., Clavera, I., Hoang, J., Wen, Y., Langlois, E., Zhang, S., Zhang, G., Abbeel, P., and Ba, J. (2019) · 1907
Earlier work this paper cites.
Multi-Step Greedy and Approximate Real Time Dynamic Programming
Original
Efroni, Y., Ghavamzadeh, M., and Mannor, S. (2019a) · 1909
Earlier work this paper cites.
Generalization of reinforcement learners with working and episodic memory
Original
Fortunato, M., Tan, M., Faulkner, R., Hansen, S., Badia, A. P., Buttimore, G., Deck, C., Leibo, J. Z., and Blundell, C. (2019) · 1910
Earlier work this paper cites.
Mastering atari, go, chess and shogi by planning with a learned model
Original
Schrittwieser, J., Antonoglou, I., Hubert, T., Simonyan, K., Sifre, L., Schmitt, S., Guez, A., Lockhart, E., Hassabis, D., Graepel, T., et al. (2019) · 1911
Earlier work this paper cites.
Adaptive Online Planning for Continual Lifelong Learning
Original
Lu, K., Mordatch, I., and Abbeel, P. (2019) · 1912
Earlier work this paper cites.
On the likelihood that one unknown probability exceeds another in view of the evidence of two samples
Thompson, W. R. (1933) · 1933
Earlier work this paper cites.
The Nature of Explanation
Craik, K. J. W. (1943) · 1943
Earlier work this paper cites.
Bootstrapping from game tree search
Veness, J., Silver, D., Blair, A., and Uther, W. (2009) · 1945
Earlier work this paper cites.
Cognitive maps in rats and men
Tolman, E. C. (1948) · 1948
Earlier work this paper cites.
Dynamic programming
Bellman, R. (1966) · 1966
Earlier work this paper cites.
The development and generalization of” contingency awareness” in early infancy: Some hypotheses
Watson, J. S. (1966) · 1966
Earlier work this paper cites.
Some studies in machine learning using the game of checkers. II - Recent progress
Samuel, A. L. (1967) · 1967
Earlier work this paper cites.
System identification—a survey
Åström, K. J. and Eykhoff, P. (1971) · 1971
Earlier work this paper cites.
Heuristic and analytic processes in reasoning
Evans, J. S. B. (1984) · 1984
Earlier work this paper cites.
Asymptotically efficient adaptive allocation rules
Lai, T. L. and Robbins, H. (1985) · 1985
Earlier work this paper cites.
Neural networks for control and system identification
Werbos, P. J. (1989) · 1989
Earlier work this paper cites.
Real-time heuristic search
Korf, R. E. (1990) · 1990
Earlier work this paper cites.
Identification and control of dynamical systems using neural networks
Narendra, K. S. and Parthasarathy, K. (1990) · 1990
Earlier work this paper cites.
Integrated architectures for learning, planning, and reacting based on approximating dynamic programming
Sutton, R. S. (1990) · 1990
Earlier work this paper cites.
Curious model-building control systems
Schmidhuber, J. (1991) · 1991
Earlier work this paper cites.
Dyna, an integrated architecture for learning, planning, and reacting
Sutton, R. S. (1991) · 1991
Earlier work this paper cites.
Reinforcement learning with perceptual aliasing: The perceptual distinctions approach
Chrisman, L. (1992) · 1992
Earlier work this paper cites.
Forward models: Supervised learning with a distal teacher
Jordan, M. I. and Rumelhart, D. E. (1992) · 1992
Earlier work this paper cites.
Self-improving reactive agents based on reinforcement learning, planning and teaching
Lin, L.-J. (1992) · 1992
Earlier work this paper cites.
Memory approaches to reinforcement learning in non-Markovian domains
Lin, L.-J. and Mitchell, T. M. (1992) · 1992
Earlier work this paper cites.
Efficient exploration in reinforcement learning
Thrun, S. B. (1992) · 1992
Earlier work this paper cites.
Q-learning
Watkins, C. J. and Dayan, P. (1992) · 1992
Earlier work this paper cites.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Williams, R. J. (1992) · 1992
Earlier work this paper cites.
Improving generalization for temporal difference learning: The successor representation
Dayan, P. (1993) · 1993
Earlier work this paper cites.
Feudal reinforcement learning
Dayan, P. and Hinton, G. E. (1993) · 1993
Earlier work this paper cites.
Reinforcement learning for robots using neural networks
Lin, L.-J. (1993) · 1993
Earlier work this paper cites.
Prioritized sweeping: Reinforcement learning with less data and less time
Moore, A. W. and Atkeson, C. G. (1993) · 1993
Earlier work this paper cites.
Application of the recurrent multilayer perceptron in modeling complex process dynamics
Parlos, A. G., Chong, K. T., and Atiya, A. F. (1994) · 1994
Earlier work this paper cites.
On-line Q-learning using connectionist systems
Rummery, G. A. and Niranjan, M. (1994) · 1994
Earlier work this paper cites.
Learning to act using real-time dynamic programming
Barto, A. G., Bradtke, S. J., and Singh, S. P. (1995) · 1995
Earlier work this paper cites.
Dynamic programming and optimal control
Bertsekas, D. P., Bertsekas, D. P., Bertsekas, D. P., and Bertsekas, D. P. (1995) · 1995
Earlier work this paper cites.
Reinforcement learning with soft state aggregation
Singh, S. P., Jaakkola, T., and Jordan, M. I. (1995) · 1995
Earlier work this paper cites.
Finding structure in reinforcement learning
Thrun, S. and Schwartz, A. (1995) · 1995
Earlier work this paper cites.
An internal model for sensorimotor integration
Wolpert, D. M., Ghahramani, Z., and Jordan, M. I. (1995) · 1995
Earlier work this paper cites.
Hierarchical recurrent neural networks for long-term dependencies
El Hihi, S. and Bengio, Y. (1996) · 1996
Earlier work this paper cites.
Parameter estimation for linear dynamical systems
Ghahramani, Z. and Hinton, G. E. (1996) · 1996
Earlier work this paper cites.
Locally weighted learning for control
Atkeson, C. G., Moore, A. W., and Schaal, S. (1997) · 1997
Earlier work this paper cites.
A comparison of direct and model-based reinforcement learning
Atkeson, C. G. and Santamaria, J. C. (1997) · 1997
Earlier work this paper cites.
Efficient reinforcement learning: Model-based acrobot control
Boone, G. (1997) · 1997
Earlier work this paper cites.
Multitask learning
Caruana, R. (1997) · 1997
Earlier work this paper cites.
Long short-term memory
Hochreiter, S. and Schmidhuber, J. (1997) · 1997
Earlier work this paper cites.
Reinforcement learning with selective perception and hidden state
McCallum, R. (1997) · 1997
Earlier work this paper cites.
Predicting time series with support vector machines
Müller, K.-R., Smola, A. J., Rätsch, G., Schölkopf, B., Kohlmorgen, J., and Vapnik, V. (1997) · 1997
Earlier work this paper cites.
On-line Policy Improvement using Monte-Carlo Search
Tesauro, G. and Galperin, G. R. (1997) · 1997
Earlier work this paper cites.
Bayesian Q-learning
Dearden, R., Friedman, N., and Russell, S. (1998) · 1998
Earlier work this paper cites.
Rapidly-exploring random trees: A new tool for path planning
LaValle, S. M. (1998) · 1998
Earlier work this paper cites.
Efficient model-based exploration
Wiering, M. and Schmidhuber, J. (1998) · 1998
Earlier work this paper cites.
TDLeaf (lambda): Combining temporal difference learning with game-tree search
Baxter, J., Tridgell, A., and Weaver, L. (1999) · 1999
Earlier work this paper cites.
Model based Bayesian exploration
Dearden, R., Friedman, N., and Andre, D. (1999) · 1999
Earlier work this paper cites.
Learning nonlinear dynamical systems using an EM algorithm
Ghahramani, Z. and Roweis, S. T. (1999) · 1999
Earlier work this paper cites.
Learning Policies with External Memory
Peshkin, L., Meuleau, N., and Kaelbling, L. P. (1999) · 1999
Earlier work this paper cites.
Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning
Sutton, R. S., Precup, D., and Singh, S. (1999) · 1999
Earlier work this paper cites.
Autonomous helicopter control using reinforcement learning policy search methods
Bagnell, J. A. and Schneider, J. G. (2001) · 2001
Earlier work this paper cites.
System identification
Ljung, L. (2001) · 2001
Earlier work this paper cites.
Automatic discovery of subgoals in reinforcement learning using diverse density
McGovern, A. and Barto, A. G. (2001) · 2001
Earlier work this paper cites.
Using confidence bounds for exploitation-exploration trade-offs
Auer, P. (2002) · 2002
Earlier work this paper cites.
R-max-a general polynomial time algorithm for near-optimal reinforcement learning
Brafman, R. I. and Tennenholtz, M. (2002) · 2002
Earlier work this paper cites.
Multiple model-based reinforcement learning
Doya, K., Samejima, K., Katagiri, K.-i., and Kawato, M. (2002) · 2002
Earlier work this paper cites.
Optimal Learning: Computational procedures for Bayes-adaptive Markov decision processes
Duff, M. O. and Barto, A. (2002) · 2002
Earlier work this paper cites.
Approximately optimal approximate reinforcement learning
Kakade, S. and Langford, J. (2002) · 2002
Earlier work this paper cites.
Q-cut—dynamic discovery of sub-goals in reinforcement learning
Menache, I., Mannor, S., and Shimkin, N. (2002) · 2002
Earlier work this paper cites.
Recent advances in hierarchical reinforcement learning
Barto, A. G. and Mahadevan, S. (2003) · 2003
Earlier work this paper cites.
Subgoal discovery for hierarchical reinforcement learning using learned policies
Goel, S. and Huber, M. (2003) · 2003
Earlier work this paper cites.
Generalizing plans to new environments in relational MDPs
Guestrin, C., Koller, D., Gearhart, C., and Kanodia, N. (2003) · 2003
Earlier work this paper cites.
On the sample complexity of reinforcement learning
Kakade, S. M. et al. (2003) · 2003
Earlier work this paper cites.
Error bounds for approximate policy iteration
Munos, R. (2003) · 2003
Earlier work this paper cites.
Dynamic abstraction in reinforcement learning via clustering
Mannor, S., Menache, I., Hoze, A., and Klein, U. (2004) · 2004
Earlier work this paper cites.
A point-based POMDP algorithm for robot planning
Spaan, M. T. and Spaan, N. (2004) · 2004
Earlier work this paper cites.
Learning first-order Markov models for control
Abbeel, P. and Ng, A. Y. (2005) · 2005
Earlier work this paper cites.
Intrinsically motivated reinforcement learning
Chentanez, N., Barto, A. G., and Singh, S. P. (2005) · 2005
Earlier work this paper cites.
Learning in non-stationary partially observable Markov decision processes
Jaulmes, R., Pineau, J., and Precup, D. (2005) · 2005
Earlier work this paper cites.
Think Too Fast Nor Too Slow: The Computational Trade-off Between Planning And Reinforcement Learning
Original
Moerland, T. M., Deichler, A., Baldi, S., Broekens, J., and Jonker, C. M. (2020b) · 2005
Earlier work this paper cites.
Planning to Explore via Self-Supervised World Models
Original
Sekar, R., Rybkin, O., Daniilidis, K., Abbeel, P., Hafner, D., and Pathak, D. (2020) · 2005
Earlier work this paper cites.
Identifying useful subgoals in reinforcement learning by local graph partitioning
Şimşek, Ö., Wolfe, A. P., and Barto, A. G. (2005) · 2005
Earlier work this paper cites.
A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems
Todorov, E. and Li, W. (2005) · 2005
Earlier work this paper cites.
Pattern recognition and machine learning
Bishop, C. M. (2006) · 2006
Earlier work this paper cites.
Dealing with non-stationary environments using context detection
Da Silva, B. C., Basso, E. W., Bazzan, A. L., and Engel, P. M. (2006) · 2006
Earlier work this paper cites.
A framework for transfer in reinforcement learning
Konidaris, G. D. (2006) · 2006
Earlier work this paper cites.
A Framework for Reinforcement Learning and Planning
Original
Moerland, T. M., Broekens, J., and Jonker, C. M. (2020a) · 2006
Earlier work this paper cites.
Pac reinforcement learning bounds for rtdp and rand-rtdp
Strehl, A. L., Li, L., and Littman, M. L. (2006) · 2006
Earlier work this paper cites.
Gaussian process dynamical models
Wang, J., Hertzmann, A., and Fleet, D. J. (2006) · 2006
Earlier work this paper cites.
Using Linear Programming for Bayesian Exploration in Markov Decision Processes
Castro, P. S. and Precup, D. (2007) · 2007
Earlier work this paper cites.
Model-based function approximation in reinforcement learning
Jong, N. K. and Stone, P. (2007) · 2007
Earlier work this paper cites.
Building Portable Options: Skill Transfer in Reinforcement Learning
Konidaris, G. and Barto, A. G. (2007) · 2007
Earlier work this paper cites.
Intrinsic motivation systems for autonomous mental development
Oudeyer, P.-Y., Kaplan, F., and Hafner, V. V. (2007) · 2007
Earlier work this paper cites.
Core knowledge
Spelke, E. S. and Kinzler, K. D. (2007) · 2007
Earlier work this paper cites.
Multi-task reinforcement learning: a hierarchical Bayesian approach
Wilson, A., Fern, A., Ray, S., and Tadepalli, P. (2007) · 2007
Earlier work this paper cites.
An object-oriented representation for efficient reinforcement learning
Diuk, C., Cohen, A., and Littman, M. L. (2008) · 2008
Earlier work this paper cites.
Sarsop: Efficient point-based pomdp planning by approximating optimally reachable belief spaces
Kurniawati, H., Hsu, D., and Lee, W. S. (2008) · 2008
Earlier work this paper cites.
How can we define intrinsic motivation
Oudeyer, P.-Y., Kaplan, F., et al. (2008) · 2008
Earlier work this paper cites.
An analysis of linear models, linear value-function approximation, and feature selection for reinforcement learning
Parr, R., Li, L., Taylor, G., Painter-Wakefield, C., and Littman, M. L. (2008) · 2008
Earlier work this paper cites.

Similar

Model-Based Deep Reinforcement Learning for High-Dimensional Problems, a Survey
Original
Plaat, A., Kosters, W., and Preuss, M. (2020) · 2008
Cited alongside, same era.
Sample-based learning and search with permanent and transient memories
Silver, D., Sutton, R. S., and Müller, M. (2008) · 2008
Cited alongside, same era.
Dyna-style Planning with Linear Function Approximation and Prioritized Sweeping
Sutton, R. S., Szepesvári, C., Geramifard, A., and Bowling, M. (2008) · 2008
Cited alongside, same era.
A Bayesian sampling approach to exploration in reinforcement learning
Asmuth, J., Li, L., Littman, M. L., Nouri, A., and Wingate, D. (2009) · 2009
Cited alongside, same era.
R-iac: Robust intrinsically motivated exploration and active learning
Baranes, A. and Oudeyer, P.-Y. (2009) · 2009
Cited alongside, same era.
Curriculum learning
Bengio, Y., Louradour, J., Collobert, R., and Weston, J. (2009) · 2009
Cited alongside, same era.
Aleatory or epistemic? Does it matter?
Der Kiureghian, A. and Ditlevsen, O. (2009) · 2009
Cited alongside, same era.
Near-Bayesian exploration in polynomial time
Kolter, J. Z. and Ng, A. Y. (2009) · 2009
Cited alongside, same era.
Learning Representation and Control in Markov Decision Processes: New Frontiers
Mahadevan, S. (2009) · 2009
Cited alongside, same era.
Transfer learning for reinforcement learning domains: A survey
Taylor, M. E. and Stone, P. (2009) · 2009
Cited alongside, same era.
Transfer Learning in Deep Reinforcement Learning: A Survey
Original
Zhu, Z., Lin, K., and Zhou, J. (2020) · 2009
Cited alongside, same era.
Near-optimal Regret Bounds for Reinforcement Learning
Jaksch, T., Ortner, R., and Auer, P. (2010) · 2010
Cited alongside, same era.

Then

Safe model-based reinforcement learning with stability guarantees
Berkenkamp, F., Turchetta, M., Schoellig, A., and Krause, A. (2017) · 2017
Later among the works it cites.
Recurrent environment simulators
Original
Chiappa, S., Racaniere, S., Wierstra, D., and Mohamed, S. (2017) · 2017
Later among the works it cites.
Stochastic neural networks for hierarchical reinforcement learning
Original
Florensa, C., Duan, Y., and Abbeel, P. (2017) · 2017
Later among the works it cites.
Multi-level discovery of deep options
Original
Fox, R., Krishnan, S., Stoica, I., and Goldberg, K. (2017) · 2017
Later among the works it cites.
A disentangled recognition and nonlinear dynamics model for unsupervised learning
Fraccaro, M., Kamronn, S., Paquet, U., and Winther, O. (2017) · 2017
Later among the works it cites.
Generative temporal models with memory
Original
Gemici, M., Hung, C.-C., Santoro, A., Wayne, G., Mohamed, S., Rezende, D. J., Amos, D., and Lillicrap, T. (2017) · 2017
Later among the works it cites.
Reinforcement learning and episodic memory in humans and animals: an integrative framework
Gershman, S. J. and Daw, N. D. (2017) · 2017
Later among the works it cites.
Metacontrol for adaptive imagination-based optimization
Original
Hamrick, J. B., Ballard, A. J., Pascanu, R., Vinyals, O., Heess, N., and Battaglia, P. W. (2017) · 2017
Later among the works it cites.
Hierarchical reinforcement learning
Hengst, B. (2017) · 2017
Later among the works it cites.
Video pixel networks
Kalchbrenner, N., van den Oord, A., Simonyan, K., Danihelka, I., Vinyals, O., Graves, A., and Kavukcuoglu, K. (2017) · 2017
Later among the works it cites.
Uncertainty-driven imagination for continuous deep reinforcement learning
Kalweit, G. and Boedecker, J. (2017) · 2017
Later among the works it cites.
Data-efficient reinforcement learning with probabilistic model predictive control
Original
Kamthe, S. and Deisenroth, M. P. (2017) · 2017
Later among the works it cites.
Schema networks: Zero-shot transfer with a generative causal model of intuitive physics
Kansky, K., Silver, T., Mély, D. A., Eldawy, M., Lázaro-Gredilla, M., Lou, X., Dorfman, N., Sidor, S., Phoenix, S., and George, D. (2017) · 2017
Later among the works it cites.
A laplacian framework for option discovery in reinforcement learning
Machado, M. C., Bellemare, M. G., and Bowling, M. (2017) · 2017
Later among the works it cites.
Teacher-student curriculum learning
Original
Matiisen, T., Oliver, A., Cohen, T., and Schulman, J. (2017) · 2017
Later among the works it cites.
Prediction and control with temporal segment models
Mishra, N., Abbeel, P., and Mordatch, I. (2017) · 2017
Later among the works it cites.
The successor representation in human reinforcement learning
Momennejad, I., Russek, E. M., Cheong, J. H., Botvinick, M. M., Daw, N. D., and Gershman, S. J. (2017) · 2017
Later among the works it cites.
Value prediction network
Oh, J., Singh, S., and Lee, H. (2017) · 2017
Later among the works it cites.
Why is posterior sampling better than optimism for reinforcement learning?
Osband, I. and Van Roy, B. (2017) · 2017
Later among the works it cites.
Count-based exploration with neural density models
Ostrovski, G., Bellemare, M. G., van den Oord, A., and Munos, R. (2017) · 2017
Later among the works it cites.
Learning model-based planning from scratch
Original
Pascanu, R., Li, Y., Vinyals, O., Heess, N., Buesing, L., Racanière, S., Reichert, D., Weber, T., Wierstra, D., and Battaglia, P. (2017) · 2017
Later among the works it cites.
Curiosity-driven exploration by self-supervised prediction
Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T. (2017) · 2017
Later among the works it cites.
Parameter space noise for exploration
Original
Plappert, M., Houthooft, R., Dhariwal, P., Sidor, S., Chen, R. Y., Chen, X., Asfour, T., Abbeel, P., and Andrychowicz, M. (2017) · 2017
Later among the works it cites.
Survey of model-based reinforcement learning: Applications on robotics
Polydoros, A. S. and Nalpantidis, L. (2017) · 2017
Later among the works it cites.
Neural Episodic Control
Pritzel, A., Uria, B., Srinivasan, S., Badia, A. P., Vinyals, O., Hassabis, D., Wierstra, D., and Blundell, C. (2017) · 2017
Later among the works it cites.
Imagination-augmented agents for deep reinforcement learning
Racanière, S., Weber, T., Reichert, D., Buesing, L., Guez, A., Rezende, D. J., Badia, A. P., Vinyals, O., Heess, N., Li, Y., et al. (2017) · 2017
Later among the works it cites.
Multi-objective decision making
Roijers, D. M. and Whiteson, S. (2017) · 2017
Later among the works it cites.
Hierarchical and interpretable skill acquisition in multi-task reinforcement learning
Original
Shu, T., Xiong, C., and Socher, R. (2017) · 2017
Later among the works it cites.
Self-correcting models for model-based reinforcement learning
Talvitie, E. (2017) · 2017
Later among the works it cites.
A deep hierarchical approach to lifelong learning in minecraft
Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J., and Mannor, S. (2017) · 2017
Later among the works it cites.
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P. (2017) · 2017
Later among the works it cites.
Feudal networks for hierarchical reinforcement learning
Vezhnevets, A. S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., and Kavukcuoglu, K. (2017) · 2017
Later among the works it cites.
Seqgan: Sequence generative adversarial nets with policy gradient
Yu, L., Zhang, W., Wang, J., and Yu, Y. (2017) · 2017
Later among the works it cites.
Variational option discovery algorithms
Original
Achiam, J., Edwards, H., Amodei, D., and Abbeel, P. (2018) · 2018
Later among the works it cites.
Towards a Simple Approach to Multi-step Model-based Reinforcement Learning
Original
Asadi, K., Cater, E., Misra, D., and Littman, M. L. (2018) · 2018
Later among the works it cites.
Relational inductive biases, deep learning, and graph networks
Original
Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., et al. (2018) · 2018
Later among the works it cites.
Sample-efficient reinforcement learning with stochastic ensemble value expansion
Buckman, J., Hafner, D., Tucker, G., Brevdo, E., and Lee, H. (2018) · 2018
Later among the works it cites.
Learning and querying fast generative models for reinforcement learning
Original
Buesing, L., Weber, T., Racaniere, S., Eslami, S., Rezende, D., Reichert, D. P., Viola, F., Besse, F., Gregor, K., Hassabis, D., et al. (2018) · 2018
Later among the works it cites.
Contingency-aware exploration in reinforcement learning
Original
Choi, J., Guo, Y., Moczulski, M., Oh, J., Wu, N., Norouzi, M., and Lee, H. (2018) · 2018
Later among the works it cites.
Deep reinforcement learning in a handful of trials using probabilistic dynamics models
Chua, K., Calandra, R., McAllister, R., and Levine, S. (2018) · 2018
Later among the works it cites.
Model-Based Reinforcement Learning via Meta-Policy Optimization
Clavera, I., Rothfuss, J., Schulman, J., Fujita, Y., Asfour, T., and Abbeel, P. (2018) · 2018
Later among the works it cites.
Efficient model-based deep reinforcement learning with variational state tabulation
Original
Corneil, D., Gerstner, W., and Brea, J. (2018) · 2018
Later among the works it cites.
End-to-end differentiable physics for learning and control
de Avila Belbute-Peres, F., Smith, K., Allen, K., Tenenbaum, J., and Kolter, J. Z. (2018) · 2018
Later among the works it cites.
Forward-backward reinforcement learning
Original
Edwards, A. D., Downs, L., and Davidson, J. C. (2018) · 2018
Later among the works it cites.
Beyond the One-Step Greedy Approach in Reinforcement Learning
Efroni, Y., Dalal, G., Scherrer, B., and Mannor, S. (2018) · 2018
Later among the works it cites.
Treeqn and atreec: Differentiable tree planning for deep reinforcement learning
Farquhar, G., Rocktäschel, T., Igl, M., and Whiteson, S. (2018) · 2018
Later among the works it cites.
Automatic Goal Generation for Reinforcement Learning Agents
Florensa, C., Held, D., Geng, X., and Abbeel, P. (2018) · 2018
Later among the works it cites.
Meta learning shared hierarchies
Frans, K., Ho, J., Chen, X., Abbeel, P., and Schulman, J. (2018) · 2018
Later among the works it cites.
Learning Actionable Representations with Goal-Conditioned Policies
Original
Ghosh, D., Gupta, A., and Levine, S. (2018) · 2018
Later among the works it cites.
Learning to search with MCTSnets
Original
Guez, A., Weber, T., Antonoglou, I., Simonyan, K., Vinyals, O., Wierstra, D., Munos, R., and Silver, D. (2018) · 2018
Later among the works it cites.
Recurrent world models facilitate policy evolution
Ha, D. and Schmidhuber, J. (2018) · 2018
Later among the works it cites.
Learning an Embedding Space for Transferable Robot Skills
Hausman, K., Springenberg, J. T., Wang, Z., Heess, N., and Riedmiller, M. (2018) · 2018
Later among the works it cites.
Deep reinforcement learning that matters
Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D. (2018) · 2018
Later among the works it cites.
The effect of planning shape on dyna-style planning in high-dimensional state spaces
Original
Holland, G. Z., Talvitie, E. J., and Bowling, M. (2018) · 2018
Later among the works it cites.
Time-agnostic prediction: Predicting predictable video frames
Original
Jayaraman, D., Ebert, F., Efros, A. A., and Levine, S. (2018) · 2018
Later among the works it cites.
Feedback-Based Tree Search for Reinforcement Learning
Jiang, D., Ekwedike, E., and Liu, H. (2018) · 2018
Later among the works it cites.
Is Q-learning provably efficient?
Jin, C., Allen-Zhu, Z., Bubeck, S., and Jordan, M. I. (2018) · 2018
Later among the works it cites.
Variance reduction methods for sublinear reinforcement learning
Kakade, S., Wang, M., and Yang, L. F. (2018) · 2018
Later among the works it cites.
Learning plannable representations with causal infogan
Kurutach, T., Tamar, A., Yang, G., Russell, S. J., and Abbeel, P. (2018) · 2018
Later among the works it cites.
Curiosity Driven Exploration of Learned Disentangled Goal Spaces
Laversanne-Finot, A., Pere, A., and Oudeyer, P.-Y. (2018) · 2018
Later among the works it cites.
State representation learning for control: An overview
Lesort, T., Díaz-Rodríguez, N., Goudou, J.-F., and Filliat, D. (2018) · 2018
Later among the works it cites.
Episodic memory deep Q-networks
Original
Lin, Z., Zhao, T., Yang, G., and Zhang, L. (2018) · 2018
Later among the works it cites.
Plan online, learn offline: Efficient learning and exploration via model-based control
Original
Lowrey, K., Rajeswaran, A., Kakade, S., Todorov, E., and Mordatch, I. (2018) · 2018
Later among the works it cites.
Now I Remember! Episodic Memory For Reinforcement Learning
Loynd, R., Hausknecht, M., Li, L., and Deng, L. (2018) · 2018
Later among the works it cites.
Revisiting the arcade learning environment: Evaluation protocols and open problems for general agents
Machado, M. C., Bellemare, M. G., Talvitie, E., Veness, J., Hausknecht, M., and Bowling, M. (2018) · 2018
Later among the works it cites.
Data-efficient hierarchical reinforcement learning
Nachum, O., Gu, S. S., Lee, H., and Levine, S. (2018) · 2018
Later among the works it cites.
Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning
Nagabandi, A., Kahn, G., Fearing, R. S., and Levine, S. (2018c) · 2018
Later among the works it cites.
Adaptive skip intervals: Temporal abstraction for recurrent dynamical models
Neitz, A., Parascandolo, G., Bauer, S., and Schölkopf, B. (2018) · 2018
Later among the works it cites.
Unsupervised learning of goal spaces for intrinsically motivated goal exploration
Original
Péré, A., Forestier, S., Sigaud, O., and Oudeyer, P.-Y. (2018) · 2018
Later among the works it cites.
Temporal Difference Models: Model-Free Deep RL for Model-Based Control
Pong, V., Gu, S., Dalal, M., and Levine, S. (2018) · 2018
Later among the works it cites.
Learning abstract options
Riemer, M., Liu, M., and Tesauro, G. (2018) · 2018
Later among the works it cites.
Disentangling Controllable and Uncontrollable Factors of Variation by Interacting with the World
Original
Sawada, Y. (2018) · 2018
Later among the works it cites.
Time-contrastive networks: Self-supervised learning from video
Sermanet, P., Lynch, C., Chebotar, Y., Hsu, J., Jang, E., Schaal, S., Levine, S., and Brain, G. (2018) · 2018
Later among the works it cites.
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · 2018
Later among the works it cites.
Universal planning networks
Original
Srinivas, A., Jabri, A., Abbeel, P., Levine, S., and Finn, C. (2018) · 2018
Later among the works it cites.
Reinforcement learning: An introduction
Sutton, R. S. and Barto, A. G. (2018) · 2018
Later among the works it cites.
Disentangling the independently controllable factors of variation by interacting with the world
Original
Thomas, V., Bengio, E., Fedus, W., Pondard, J., Beaudoin, P., Larochelle, H., Pineau, J., Precup, D., and Bengio, Y. (2018) · 2018
Later among the works it cites.
Contrastive explanations for reinforcement learning in terms of expected consequences
Original
van der Waa, J., van Diggelen, J., Bosch, K. v. d., and Neerincx, M. (2018) · 2018
Later among the works it cites.
Relational neural expectation maximization: Unsupervised discovery of objects and their interactions
Original
Van Steenkiste, S., Chang, M., Greff, K., and Schmidhuber, J. (2018) · 2018
Later among the works it cites.
Solving the Rubik’s cube with deep reinforcement learning and search
Agostinelli, F., McAleer, S., Shmakov, A., and Baldi, P. (2019) · 2019
Later among the works it cites.
Unsupervised state representation learning in atari
Anand, A., Racah, E., Ozair, S., Bengio, Y., Côté, M.-A., and Hjelm, R. D. (2019) · 2019
Later among the works it cites.
A differentiable physics engine for deep learning in robotics
Degrave, J., Hermans, M., Dambre, J., et al. (2019) · 2019
Later among the works it cites.
Feature control as intrinsic motivation for hierarchical reinforcement learning
Dilokthanakul, N., Kaplanis, C., Pawlowski, N., and Shanahan, M. (2019) · 2019
Later among the works it cites.
Combined reinforcement learning via abstract representations
François-Lavet, V., Bengio, Y., Precup, D., and Pineau, J. (2019) · 2019
Later among the works it cites.
An Investigation of Model-Free Planning
Guez, A., Mirza, M., Gregor, K., Kabra, R., Racaniere, S., Weber, T., Raposo, D., Santoro, A., Orseau, L., Eccles, T., et al. (2019) · 2019
Later among the works it cites.
Analogues of mental simulation and imagination in deep learning
Hamrick, J. B. (2019) · 2019
Later among the works it cites.
When to trust your model: Model-based policy optimization
Janner, M., Fu, J., Zhang, M., and Levine, S. (2019) · 2019
Later among the works it cites.
Hierarchical Reinforcement Learning with Hindsight
Levy, A., Platt, R., and Saenko, K. (2019) · 2019
Later among the works it cites.
Behaviour Suite for Reinforcement Learning
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepezvari, C., Singh, S., Roy, B. V., Sutton, R., Silver, D., and Hasselt, H. V. (2019) · 2019
Later among the works it cites.
Dynamics-Aware Unsupervised Discovery of Skills
Sharma, A., Gu, S., Levine, S., Kumar, V., and Hausman, K. (2019) · 2019
Later among the works it cites.
Model-Based Active Exploration
Shyam, P., Jaśkowski, W., and Gomez, F. (2019) · 2019
Later among the works it cites.
When to use parametric models in reinforcement learning?
van Hasselt, H. P., Hessel, M., and Aslanides, J. (2019) · 2019
Later among the works it cites.
Meta-learning
Vanschoren, J. (2019) · 2019
Later among the works it cites.
Model-Based Multi-objective Reinforcement Learning with Unknown Weights
Yamaguchi, T., Nagahama, S., Ichikawa, Y., and Takadama, K. (2019) · 2019
Later among the works it cites.
Tighter problem-dependent regret bounds in reinforcement learning without domain knowledge using value function bounds
Zanette, A. and Brunskill, E. (2019) · 2019
Later among the works it cites.
SOLAR: Deep Structured Representations for Model-Based Reinforcement Learning
Zhang, M., Vikram, S., Smith, L., Abbeel, P., Johnson, M., and Levine, S. (2019) · 2019
Later among the works it cites.
Amrl: Aggregated memory for reinforcement learning
Beck, J., Ciosek, K., Devlin, S., Tschiatschek, S., Zhang, C., and Hofmann, K. (2020) · 2020
Closest in time.
The Value Equivalence Principle for Model-Based Reinforcement Learning
Grimm, C., Barreto, A., Singh, S., and Silver, D. (2020) · 2020
Closest in time.
Combining q-learning and search with amortized value estimates
Hamrick, J. B., Bapst, V., Sanchez-Gonzalez, A., Pfaff, T., Weber, T., Buesing, L., and Battaglia, P. W. (2020) · 2020
Closest in time.
Contrastive Learning of Structured World Models
Kipf, T., van der Pol, E., and Welling, M. (2020) · 2020
Closest in time.
The LoCA Regret: A Consistent Metric to Evaluate Model-Based Behavior in Reinforcement Learning
Van Seijen, H., Nekoei, H., Racah, E., and Chandar, S. (2020) · 2020
Closest in time.
Value targets in off-policy AlphaZero: a new greedy backup
Willemsen, D., Baier, H., and Kaisers, M. (2020) · 2020
Closest in time.
Generalizable episodic memory for deep reinforcement learning
Original
Hu, H., Ye, J., Zhu, G., Ren, Z., and Zhang, C. (2021) · 2021
Closest in time.
World model as a graph: Learning latent landmarks for planning
Zhang, L., Yang, G., and Stadie, B. C. (2021) · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…