Towards Practical Credit Assignment for Deep Reinforcement Learning

Simple statistical gradient-following algorithms for connectionist reinforcement learning

Williams, R. J · 1992

Earlier work this paper cites.

Policy invariance under reward transformations: Theory and application to reward shaping

Ng, A. Y., Harada, D., and Russell, S · 1999

Earlier work this paper cites.

Policy gradient methods for reinforcement learning with function approximation

Sutton, R. S., McAllester, D. A., Singh, S. P., Mansour, Y., et al · 1999

Earlier work this paper cites.

Counterfactual credit assignment in model-free reinforcement learning

Original

Mesnard, T., Weber, T., Viola, F., Thakoor, S., Saade, A., Harutyunyan, A., Dabney, W., Stepleton, T., Heess, N., Guez, A., Hutter, M., Buesing, L., and Munos, R · 2011

Earlier work this paper cites.

The arcade learning environment: An evaluation platform for general agents

Original

Bellemare, M. G., Naddaf, Y., Veness, J., and Bowling, M · 2012

Earlier work this paper cites.

Markov decision processes: discrete stochastic dynamic programming

Puterman, M. L · 2014

Earlier work this paper cites.

Human-level control through deep reinforcement learning

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al · 2015

Earlier work this paper cites.

Openai gym, 2016

Original

Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W · 2016

Earlier work this paper cites.

Asynchronous methods for deep reinforcement learning

Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K · 2016

Cited alongside, same era.

Mastering the game of go with deep neural networks and tree search

Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T., Leach, M., Kavukcuoglu, K., Graepel, T., and Hassabis, D · 2016

Cited alongside, same era.

Pytorch implementations of reinforcement learning algorithms

Kostrikov, I · 2018

Cited alongside, same era.

Data center cooling using model-predictive control

Lazic, N., Boutilier, C., Lu, T., Wong, E., Roy, B., Ryu, M., and Imwalle, G · 2018

Cited alongside, same era.

Learning dexterous in-hand manipulation

Original

OpenAI, Andrychowicz, M., Baker, B., Chociej, M., Józefowicz, R., McGrew, B., Pachocki, J., Petron, A., Plappert, M., Powell, G., Ray, A., Schneider, J., Sidor, S., Tobin, J., Welinder, P., Weng, L., and Zaremba, W · 2018

On learning intrinsic rewards for policy gradient methods

Zheng, Z., Oh, J., and Singh, S · 2018

Later among the works it cites.

Rudder: Return decomposition for delayed rewards

Arjona-Medina, J. A., Gillhofer, M., Widrich, M., Unterthiner, T., Brandstetter, J., and Hochreiter, S · 2019

Later among the works it cites.

Hindsight credit assignment

Harutyunyan, A., Dabney, W., Mesnard, T., Azar, M. G., Piot, B., Heess, N., van Hasselt, H., Wayne, G., Singh, S., Precup, D., and Munos, R · 2019

Later among the works it cites.

Optimizing agent behavior over long time scales by transporting value

Hung, C.-C., Lillicrap, T., Abramson, J., Wu, Y., Mirza, M., Carnevale, F., Ahuja, A., and Wayne, G · 2019

Later among the works it cites.

Synthetic returns for long-term credit assignment, 2021

Raposo, D., Ritter, S., Santoro, A., Wayne, G., Weber, T., Botvinick, M., van Hasselt, H., and Song, F · 2021

Closest in time.

Policy gradients incorporating the future