Understand
Upside-Down Reinforcement Learning (UDRL) is an approach for solving RL problems that does not require value functions and uses only supervised learning, where the targets for given inputs in a dataset do not change over time.
- Ghosh et al.
- proved that Goal-Conditional Supervised Learning (GCSL) -- which can be viewed as a simplified version of UDRL -- optimizes a lower bound on goal-reaching performance.
- This raises expectations that such algorithms may enjoy guaranteed convergence to the optimal policy in arbitrary environments, similar to certain well-known traditional RL algorithms.
Built on
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Martin L Puterman · 2014
Earlier work this paper cites.
Hindsight experience replay, 2018
Marcin Andrychowicz, Filip Wolski, Alex Ray, Jonas Schneider, Rachel Fong, Peter Welinder, Bob McGrew, Josh Tobin, Pieter Abbeel, and Wojciech Zaremba · 2018
Earlier work this paper cites.
Similar
Learning to reach goals via iterated supervised learning, 2019
Dibya Ghosh, Abhishek Gupta, Ashwin Reddy, Justin Fu, Coline Devin, Benjamin Eysenbach, and Sergey Levine · 2019
Cited alongside, same era.
Reinforcement learning upside down: Don’t predict rewards – just map them to actions, 2019
Juergen Schmidhuber · 2019
Cited alongside, same era.
Then
Training agents using upside-down reinforcement learning, 2019
Rupesh Kumar Srivastava, Pranav Shyam, Filipe Mutz, Wojciech Jaśkowski, and Jürgen Schmidhuber · 2019
Later among the works it cites.
Reward-weighted regression converges to a global optimum
Miroslav Strupl, Francesco Faccio, Dylan R. Ashley, Rupesh Kumar Srivastava, and Jürgen Schmidhuber · 2021
Later among the works it cites.
Beyond the bibliography
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…