Fetching the paper…

Reading the bibliography…

2019

Hierarchical Deep Q-Network from Imperfect Demonstrations in Minecraft

Skrynnik, Alexey, Staroverov, Aleksey, Aitygulov, Ermek et al.

Original
PDF
Post on X

Understand

We present Hierarchical Deep Q-Network (HDQfD) that took first place in the MineRL competition.

HDQfD works on imperfect demonstrations and utilizes the hierarchical structure of expert trajectories.
We introduce the procedure of extracting an effective sequence of meta-actions and subgoals from demonstration data.
We present a structured task-dependent replay buffer and adaptive prioritizing technique that allow the HDQfD agent to gradually erase poor-quality expert data from the buffer.

Hierarchical Deep Q-Network from Imperfect Demonstrations in Minecraft · Around

Built on

The minerl competition on sample efficient reinforcement learning using human priors
Original
W. H. Guss, C. Codel, K. Hofmann, B. Houghton, N. Kuno, S. Milani, S. Mohanty, D. P. Liebana, R. Salakhutdinov, N. Topin, et al · 1904
Earlier work this paper cites.
Minerl: a large-scale dataset of minecraft demonstrations
Original
W. H. Guss, B. Houghton, N. Topin, P. Wang, C. Codel, M. Veloso, and R. Salakhutdinov · 1907
Earlier work this paper cites.
On the sample complexity of reinforcement learning
S. M. Kakade et al · 2003
Earlier work this paper cites.

Similar

Hindsight experience replay
M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welinder, B. McGrew, J. Tobin, O. P. Abbeel, and W. Zaremba · 2017
Cited alongside, same era.
Mastering the game of go without human knowledge
D. Silver, J. Schrittwieser, K. Simonyan, I. Antonoglou, A. Huang, A. Guez, T. Hubert, L. Baker, M. Lai, A. Bolton, et al · 2017
Cited alongside, same era.
Reinforcement learning from imperfect demonstrations
Original
Y. Gao, J. Lin, F. Yu, S. Levine, T. Darrell, et al · 2018
Cited alongside, same era.

Then

Deep q-learning from demonstrations
T. Hester, M. Vecerik, O. Pietquin, M. Lanctot, T. Schaul, B. Piot, D. Horgan, J. Quan, A. Sendonaris, I. Osband, et al · 2018
Later among the works it cites.
Overcoming exploration in reinforcement learning with demonstrations
A. Nair, B. McGrew, M. Andrychowicz, W. Zaremba, and P. Abbeel · 2018
Later among the works it cites.
Alphastar: Mastering the real-time strategy game starcraft ii
O. Vinyals, I. Babuschkin, J. Chung, M. Mathieu, M. Jaderberg, W. M. Czarnecki, A. Dudzik, A. Huang, P. Georgiev, R. Powell, et al · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…