2023

A Survey of Temporal Credit Assignment in Deep Reinforcement Learning

Pignatelli, Eduardo, Ferret, Johan, Geist, Matthieu et al.

Understand

The Credit Assignment Problem (CAP) refers to the longstanding challenge of Reinforcement Learning (RL) agents to associate actions with their long-term consequences.

Solving the CAP is a crucial step towards the successful deployment of RL in the real world since most decision problems provide feedback that is noisy, delayed, and with little or no information about the causes.
These conditions make it hard to distinguish serendipitous outcomes from those caused by informed decision-making.
However, the mathematical nature of credit and the CAP remains poorly understood and defined.

Built on

Training agents using upside-down reinforcement learning
Original
Rupesh Kumar Srivastava, Pranav Shyam, Filipe Mutz, Wojciech Jaskowski, and Jürgen Schmidhuber · 1912
Earlier work this paper cites.
Programming a computer for playing chess
Claude E Shannon · 1950
Earlier work this paper cites.
The origins of intelligence in children , volume 8
Jean Piaget, Margaret Cook, et al · 1952
Earlier work this paper cites.
Dynamic programming and Markov processes
Ronald A Howard · 1960
Earlier work this paper cites.
Steps toward artificial intelligence
Marvin Minsky · 1961
Earlier work this paper cites.
Experiments on the mechanization of game-learning part i. characterization of the model and its parameters
Donald Michie · 1963
Earlier work this paper cites.
State functions and linear control systems
Donald G Schultz · 1967
Earlier work this paper cites.
Brain function and adaptive systems: a heterostatic theory
A Harry Klopf · 1972
Earlier work this paper cites.
Markov decision processes with a new optimality criterion: Discrete time
Stratton C Jaquette · 1973
Earlier work this paper cites.
Modified policy iteration algorithms for discounted markov decision problems
Martin L Puterman and Moon Chirl Shin · 1978
Earlier work this paper cites.
The variance of discounted markov decision processes
Matthew J Sobel · 1982
Earlier work this paper cites.
Temporal credit assignment in reinforcement learning
Richard S Sutton · 1984
Earlier work this paper cites.
Learning to predict by the methods of temporal differences
Richard S Sutton · 1988
Earlier work this paper cites.
Mean, variance, and probabilistic criteria in finite markov decision processes: A review
Douglas J White · 1988
Earlier work this paper cites.
Learning from delayed rewards
Christopher J C H Watkins · 1989
Earlier work this paper cites.
Self-improving reactive agents based on reinforcement learning, planning and teaching
Long-Ji Lin · 1992
Earlier work this paper cites.
Convergence of stochastic iterative dynamic programming algorithms
Tommi Jaakkola, Michael Jordan, and Satinder Singh · 1993
Earlier work this paper cites.
Reinforcement learning with replacing eligibility traces
Satinder P Singh and Richard S Sutton · 1996
Earlier work this paper cites.
Reinforcement Learning Through Gradient Descent
Leemon C III Baird · 1999
Earlier work this paper cites.
Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning
Richard S Sutton, Doina Precup, and Satinder Singh · 1999
Earlier work this paper cites.
Models, reasoning and inference
Judea Pearl et al · 2000
Earlier work this paper cites.
Recent advances in hierarchical reinforcement learning
Andrew G Barto and Sridhar Mahadevan · 2003
Earlier work this paper cites.
All learning is local: Multi-agent learning in global reward games
Yu-Han Chang, Tracey Ho, and Leslie Kaelbling · 2003
Earlier work this paper cites.
Dueling network architectures for deep reinforcement learning
Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas · 2003
Earlier work this paper cites.
Intrinsically motivated learning of hierarchical collections of skills
Andrew G Barto, Satinder Singh, Nuttapong Chentanez, et al · 2004
Earlier work this paper cites.
Intrinsically motivated reinforcement learning
Nuttapong Chentanez, Andrew Barto, and Satinder Singh · 2004
Earlier work this paper cites.
The reward hypothesis
Richard S. Sutton · 2004
Earlier work this paper cites.
The goal construct in psychology
Andrew J Elliot and James W Fryer · 2008
Earlier work this paper cites.
Causal inference in statistics: An overview
Judea Pearl · 2009
Earlier work this paper cites.
Effects of feedback delay on learning
Hazhir Rahmandad, Nelson Repenning, and John Sterman · 2009
Earlier work this paper cites.
Where do rewards come from
Satinder Singh, Richard L Lewis, and Andrew G Barto · 2009
Earlier work this paper cites.
Using continuous action spaces to solve discrete problems
Hado van Hasselt and Marco A Wiering · 2009
Earlier work this paper cites.
Delayed reinforcement of operant behavior
Kennon A Lattal · 2010
Earlier work this paper cites.
Action-gap phenomenon in reinforcement learning
Amir-massoud Farahmand · 2011
Earlier work this paper cites.
Horde: A scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction
Richard S Sutton, Joseph Modayil, Michael Delp, Thomas Degris, Patrick M Pilarski, Adam White, and Doina Precup · 2011
Earlier work this paper cites.
Mujoco: A physics engine for model-based control
Emanuel Todorov, Tom Erez, and Yuval Tassa · 2012
Earlier work this paper cites.
Intrinsic motivation and reinforcement learning
Andrew G Barto · 2013
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling · 2013
Earlier work this paper cites.
Quantifying causal influences
Dominik Janzing, David Balduzzi, Moritz Grosse-Wentrup, and Bernhard Schölkopf · 2013
Earlier work this paper cites.
Reinforcement learning in robotics: Applications and real-world challenges
Petar Kormushev, Sylvain Calinon, and Darwin G Caldwell · 2013
Earlier work this paper cites.
Playing atari with deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller · 2013
Earlier work this paper cites.
Machine learning applications for data center optimization, 2014
Jim Gao · 2014
Earlier work this paper cites.
Off-policy learning with eligibility traces: a survey
Matthieu Geist, Bruno Scherrer, et al · 2014
Earlier work this paper cites.
Markov decision processes: discrete stochastic dynamic programming
Martin L Puterman · 2014
Earlier work this paper cites.
Hierarchical reinforcement learning: a survey
Mostafa Al-Emran · 2015
Earlier work this paper cites.
Kickback cuts backprop’s red-tape: Biologically plausible credit assignment in neural networks
David Balduzzi, Hastagiri Vanchinathan, and Joachim Buhmann · 2015
Earlier work this paper cites.
Deep reinforcement learning in large discrete action spaces
Original
Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, Theophane Weber, Thomas Degris, and Ben Coppin · 2015
Earlier work this paper cites.
A comprehensive survey on safe reinforcement learning
Javier García, Fern, and o Fernández · 2015
Earlier work this paper cites.
Learning to transduce with unbounded memory
Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom · 2015
Earlier work this paper cites.
Continuous control with deep reinforcement learning
Original
Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra · 2015
Earlier work this paper cites.
Emphatic temporal-difference learning
Original
A Rupam Mahmood, Huizhen Yu, Martha White, and Richard S Sutton · 2015
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al · 2015
Earlier work this paper cites.
Approximate modified policy iteration and its application to the game of tetris
Bruno Scherrer, Mohammad Ghavamzadeh, Victor Gabillon, Boris Lesner, and Matthieu Geist · 2015
Earlier work this paper cites.
Deep learning in neural networks: An overview
Jürgen Schmidhuber · 2015
Earlier work this paper cites.
Learning to predict independent of span
Original
Hado van Hasselt and Richard S Sutton · 2015
Earlier work this paper cites.
Deepmind lab
Original
Charles Beattie, Joel Z Leibo, Denis Teplyashin, Tom Ward, Marcus Wainwright, Heinrich Küttler, Andrew Lefrancq, Simon Green, Víctor Valdés, Amir Sadik, et al · 2016
Earlier work this paper cites.
Increasing the action gap: New operators for reinforcement learning
Marc G Bellemare, Georg Ostrovski, Arthur Guez, Philip Thomas, and Rémi Munos · 2016
Earlier work this paper cites.
Hybrid computing using a neural network with dynamic external memory
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al · 2016
Earlier work this paper cites.
The interface theory of perception
Donald D Hoffman · 2016
Earlier work this paper cites.
Vizdoom: A doom-based ai research platform for visual reinforcement learning
Michał Kempka, Marek Wydmuch, Grzegorz Runc, Jakub Toczek, and Wojciech Jaśkowski · 2016
Earlier work this paper cites.
Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 2016
Earlier work this paper cites.
High-dimensional continuous control using generalized advantage estimation
John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel · 2016
Earlier work this paper cites.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016
Earlier work this paper cites.
An emphatic approach to the problem of off-policy temporal-difference learning
Richard S Sutton, A Rupam Mahmood, and Martha White · 2016
Earlier work this paper cites.

Similar

Deep reinforcement learning with double q-learning
Hado van Hasselt, Arthur Guez, and David Silver · 2016
Cited alongside, same era.
Hindsight experience replay
Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, OpenAI Pieter Abbeel, and Wojciech Zaremba · 2017
Cited alongside, same era.
Deep reinforcement learning: A brief survey
Kai Arulkumaran, Marc Peter Deisenroth, Miles Brundage, and Anil Anthony Bharath · 2017
Cited alongside, same era.
The option-critic architecture
Pierre-Luc Bacon, Jean Harb, and Doina Precup · 2017
Cited alongside, same era.
A distributional perspective on reinforcement learning
Marc G Bellemare, Will Dabney, and Rémi Munos · 2017
Cited alongside, same era.
Reinforcement learning with deep energy-based policies
Tuomas Haarnoja, Haoran Tang, Pieter Abbeel, and Sergey Levine · 2017
Cited alongside, same era.
Reinforcement learning with unsupervised auxiliary tasks
Max Jaderberg, Volodymyr Mnih, Wojciech Marian Czarnecki, Tom Schaul, Joel Z Leibo, David Silver, and Koray Kavukcuoglu · 2017
Cited alongside, same era.
A structured self-attentive sentence embedding
Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio · 2017
Cited alongside, same era.
Time delays, competitive interdependence, and firm performance
Jukka Luoma, Sampsa Ruutu, Adelaide Wilcox King, and Henrikki Tikkanen · 2017
Cited alongside, same era.
Applying q ( λ \lambda )-learning in deep reinforcement learning to play atari games
Seyed Sajad Mousavi, Michael Schukat, Enda Howley, and Patrick Mannion · 2017
Cited alongside, same era.
Imagination-augmented agents for deep reinforcement learning
Sébastien Racanière, Théophane Weber, David Reichert, Lars Buesing, Arthur Guez, Danilo Jimenez Rezende, Adrià Puigdomènech Badia, Oriol Vinyals, Nicolas Heess, Yujia Li, et al · 2017
Cited alongside, same era.
Proximal policy optimization algorithms
Original
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017
Cited alongside, same era.

Then

Counterfactual data augmentation using locally factored dynamics
Silviu Pitis, Elliot Creager, and Animesh Garg · 2020
Later among the works it cites.
Adaptive trade-offs in off-policy learning
Mark Rowland, Will Dabney, and Rémi Munos · 2020
Later among the works it cites.
Mastering atari, go, chess and shogi by planning with a learned model
Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, et al · 2020
Later among the works it cites.
Reinforcement learning for building controls: The opportunities and challenges
Zhe Wang and Tianzhen Hong · 2020
Later among the works it cites.
Meta-gradient reinforcement learning with an objective discovered online
Zhongwen Xu, Hado P van Hasselt, Matteo Hessel, Junhyuk Oh, Satinder Singh, and David Silver · 2020
Later among the works it cites.
Self-tuning deep reinforcement learning
Original
Tom Zahavy, Zhongwen Xu, Vivek Veeriah, Matteo Hessel, Junhyuk Oh, Hado van Hasselt, David Silver, and Satinder Singh · 2020
Later among the works it cites.
Learning retrospective knowledge with reverse reinforcement learning
Shangtong Zhang, Vivek Veeriah, and Shimon Whiteson · 2020
Later among the works it cites.
Deep reinforcement learning at the edge of the statistical precipice
Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare · 2021
Later among the works it cites.
A survey of exploration methods in reinforcement learning
Original
Susan Amin, Maziar Gomrokchi, Harsh Satija, Herke van Hoof, and Doina Precup · 2021
Later among the works it cites.
An information-theoretic perspective on credit assignment in reinforcement learning
Original
Dilip Arumugam, Peter Henderson, and Pierre-Luc Bacon · 2021
Later among the works it cites.
Decision transformer: Reinforcement learning via sequence modeling
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Misha Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch · 2021
Later among the works it cites.
Parameter-based value functions
Francesco Faccio, Louis Kirsch, and Jürgen Schmidhuber · 2021
Later among the works it cites.
Self-imitation advantage learning
Johan Ferret, Olivier Pietquin, and Matthieu Geist · 2021
Later among the works it cites.
There is no turning back: A self-supervised approach for reversibility-aware reinforcement learning
Nathan Grinsztajn, Johan Ferret, Olivier Pietquin, Matthieu Geist, et al · 2021
Later among the works it cites.
Offline reinforcement learning as one big sequence modeling problem
Michael Janner, Qiyang Li, and Sergey Levine · 2021
Later among the works it cites.
Flexible option learning
Martin Klissarov and Doina Precup · 2021
Later among the works it cites.
Counterfactual credit assignment in model-free reinforcement learning
Thomas Mesnard, Theophane Weber, Fabio Viola, Shantanu Thakoor, Alaa Saade, Anna Harutyunyan, Will Dabney, Thomas S Stepleton, Nicolas Heess, Arthur Guez, et al · 2021
Later among the works it cites.
Deep reinforcement learning for cyber security
Thanh Thi Nguyen and Vijay Janapa Reddi · 2021
Later among the works it cites.
Posterior value functions: Hindsight baselines for policy gradient methods
Chris Nota, Philip Thomas, and Bruno C. Da Silva · 2021
Later among the works it cites.
Hierarchical reinforcement learning: A comprehensive survey
Shubham Pateria, Budhitama Subagdja, Ah-hwee Tan, and Chai Quek · 2021
Later among the works it cites.
Fitness beats truth in the evolution of perception
Chetan Prakash, Kyle D Stephens, Donald D Hoffman, Manish Singh, and Chris Fields · 2021
Later among the works it cites.
Synthetic returns for long-term credit assignment
David Raposo, Sam Ritter, Adam Santoro, Greg Wayne, Theophane Weber, Matt Botvinick, Hado van Hasselt, and Francis Song · 2021
Later among the works it cites.
Minihack the planet: A sandbox for open-ended reinforcement learning research
Mikayel Samvelyan, Robert Kirk, Vitaly Kurin, Jack Parker-Holder, Minqi Jiang, Eric Hambro, Fabio Petroni, Heinrich Kuttler, Edward Grefenstette, and Tim Rocktäschel · 2021
Later among the works it cites.
Hindsight expectation maximization for goal-conditioned reinforcement learning
Yunhao Tang and Alp Kucukelbir · 2021
Later among the works it cites.
Expected eligibility traces
Hado van Hasselt, Sephora Madjiheurem, Matteo Hessel, David Silver, André Barreto, and Diana Borsa · 2021
Later among the works it cites.
Offline reinforcement learning with reverse model-based imagination
Jianhao Wang, Wenzhe Li, Haozhe Jiang, Guangxiang Zhu, Siyuan Li, and Chongjie Zhang · 2021
Later among the works it cites.
Mastering atari games with limited data
Weirui Ye, Shaohuai Liu, Thanard Kurutach, Pieter Abbeel, and Yang Gao · 2021
Later among the works it cites.
All you need is supervised learning: From imitation learning to meta-rl with upside down rl
Original
Kai Arulkumaran, Dylan R Ashley, Jürgen Schmidhuber, and Rupesh K Srivastava · 2022
Later among the works it cites.
Learning relative return policies with upside-down reinforcement learning
Original
Dylan R Ashley, Kai Arulkumaran, Jürgen Schmidhuber, and Rupesh Kumar Srivastava · 2022
Later among the works it cites.
On Pearl’s Hierarchy and the Foundations of Causal Inference , pp. 507–556
Elias Bareinboim, Juan D. Correa, Duligur Ibeling, and Thomas Icard · 2022
Later among the works it cites.
Selective credit assignment
Original
Veronica Chelu, Diana Borsa, Doina Precup, and Hado van Hasselt · 2022
Later among the works it cites.
Autotelic agents with intrinsically motivated goal-conditioned reinforcement learning: a short survey
Cédric Colas, Tristan Karch, Olivier Sigaud, and Pierre-Yves Oudeyer · 2022
Later among the works it cites.
Magnetic control of tokamak plasmas through deep reinforcement learning
Jonas Degrave, Federico Felici, Jonas Buchli, Michael Neunert, Brendan Tracey, Francesco Carpanese, Timo Ewalds, Roland Hafner, Abbas Abdolmaleki, Diego de Las Casas, et al · 2022
Later among the works it cites.
Maximum entropy RL (provably) solves some robust RL problems
Benjamin Eysenbach and Sergey Levine · 2022
Later among the works it cites.
On Actions that Matter: Credit Assignment and Interpretability in Reinforcement Learning
Johan Ferret · 2022
Later among the works it cites.
Generalized decision transformer for offline hindsight information matching
Hiroki Furuta, Yutaka Matsuo, and Shixiang Shane Gu · 2022
Later among the works it cites.
Deep hierarchical planning from pixels
Danijar Hafner, Kuang-Huei Lee, Ian Fischer, and Pieter Abbeel · 2022
Later among the works it cites.
Adaptive interest for emphatic reinforcement learning
Martin Klissarov, Rasool Fakoor, Jonas Mueller, Kavosh Asadi, Taesup Kim, and Alex Smola · 2022
Later among the works it cites.
On the generalization of representations in reinforcement learning
Charline Le Lan, Stephen Tu, Adam Oberman, Rishabh Agarwal, and Marc G Bellemare · 2022
Later among the works it cites.
Multi-game decision transformers
Original
Kuang-Huei Lee, Ofir Nachum, Mengjiao Yang, Lisa Lee, Daniel Freeman, Winnie Xu, Sergio Guadarrama, Ian Fischer, Eric Jang, Henryk Michalewski, et al · 2022
Later among the works it cites.
Goal-conditioned reinforcement learning: Problems and solutions
Original
Minghuan Liu, Menghui Zhu, and Weinan Zhang · 2022
Later among the works it cites.
Direct advantage estimation
Hsiao-Ru Pan, Nico Gürtler, Alexander Neitz, and Bernhard Schölkopf · 2022
Later among the works it cites.
Mastering the game of stratego with model-free multiagent reinforcement learning
Julien Perolat, Bart De Vylder, Daniel Hennes, Eugene Tarassov, Florian Strub, Vincent de Boer, Paul Muller, Jerome T Connor, Neil Burch, Thomas Anthony, et al · 2022
Later among the works it cites.
A generalist agent
Original
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, et al · 2022
Later among the works it cites.
Learning long-term reward redistribution via randomized return decomposition
Zhizhou Ren, Ruihan Guo, Yuan Zhou, and Jian Peng · 2022
Later among the works it cites.
The phenomenon of policy churn
Original
Tom Schaul, André Barreto, John Quan, and Georg Ostrovski · 2022
Later among the works it cites.
Utility theory for sequential decision making
Mehran Shakerinava and Siamak Ravanbakhsh · 2022
Later among the works it cites.
Upside-down reinforcement learning can diverge in stochastic environments with episodic resets
Original
Miroslav Štrupl, Francesco Faccio, Dylan R Ashley, Jürgen Schmidhuber, and Rupesh Kumar Srivastava · 2022
Later among the works it cites.
How does value distribution in distributional reinforcement learning help optimization?
Original
Ke Sun, Bei Jiang, and Linglong Kong · 2022
Later among the works it cites.
Policy gradients incorporating the future
David Venuto, Elaine Lau, Doina Precup, and Ofir Nachum · 2022
Later among the works it cites.
Adversarial policies beat professional-level go ais
Original
Tony Tong Wang, Adam Gleave, Nora Belrose, Tom Tseng, Joseph Miller, Michael D Dennis, Yawen Duan, Viktor Pogrebniak, Sergey Levine, and Stuart Russell · 2022
Later among the works it cites.
Outracing champion gran turismo drivers with deep reinforcement learning
Peter R Wurman, Samuel Barrett, Kenta Kawamoto, James MacGlashan, Kaushik Subramanian, Thomas J Walsh, Roberto Capobianco, Alisa Devlic, Franziska Eckert, Florian Fuchs, et al · 2022
Later among the works it cites.
Online decision transformer
Qinqing Zheng, Amy Zhang, and Aditya Grover · 2022
Later among the works it cites.
What can learned intrinsic rewards capture?
Zeyu Zheng, Junhyuk Oh, Matteo Hessel, Zhongwen Xu, Manuel Kroiss, Hado van Hasselt, David Silver, and Satinder Singh · 2022
Later among the works it cites.
A definition of continual reinforcement learning
David Abel, Andre Barreto, Benjamin Van Roy, Doina Precup, Hado van Hasselt, and Satinder Singh · 2023
Closest in time.
Settling the reward hypothesis
Michael Bowling, John D Martin, David Abel, and Will Dabney · 2023
Closest in time.
General intelligence requires rethinking exploration
Minqi Jiang, Tim Rocktäschel, and Edward Grefenstette · 2023
Closest in time.
Human-level atari 200x faster
Steven Kapturowski, Víctor Campos, Ray Jiang, Nemanja Rakicevic, Hado van Hasselt, Charles Blundell, and Adria Puigdomenech Badia · 2023
Closest in time.
A survey of zero-shot generalisation in deep reinforcement learning
Robert Kirk, Amy Zhang, Edward Grefenstette, and Tim Rocktäschel · 2023
Closest in time.
Quantile credit assignment
Thomas Mesnard, Wenqi Chen, Alaa Saade, Yunhao Tang, Mark Rowland, Theophane Weber, Clare Lyle, Audrunas Gruslys, Michal Valko, Will Dabney, Georg Ostrovski, Eric Moulines, and Remi Munos · 2023
Closest in time.
When do transformers shine in RL? decoupling memory from credit assignment
Tianwei Ni, Michel Ma, Benjamin Eysenbach, and Pierre-Luc Bacon · 2023
Closest in time.
Distributional meta-gradient reinforcement learning
Haiyan Yin, Shuicheng YAN, and Zhongwen Xu · 2023
Closest in time.
Optimizing agent behavior over long time scales by transporting value
Chia-Chun Hung, Timothy Lillicrap, Josh Abramson, Yan Wu, Mehdi Mirza, Federico Carnevale, Arun Ahuja, and Greg Wayne · 2041
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…