Fetching the paper…

Reading the bibliography…

2022

ReIL: A Framework for Reinforced Intervention-based Imitation Learning

Parnichkun, Rom, Dailey, Matthew N., Yamashita, Atsushi

Original
PDF
Post on X

Understand

Compared to traditional imitation learning methods such as DAgger and DART, intervention-based imitation offers a more convenient and sample efficient data collection process to users.

In this paper, we introduce Reinforced Intervention-based Learning (ReIL), a framework consisting of a general intervention-based learning algorithm and a multi-task imitation learning model aimed at enabling non-expert users to train agents in real environments with little supervision or fine tuning.
ReIL achieves this with an algorithm that combines the advantages of imitation learning and reinforcement learning and a model capable of concurrently processing demonstrations, past experience, and current observations.
Experimental results from real world mobile robot navigation challenges indicate that ReIL learns rapidly from sparse supervisor corrections without suffering deterioration in performance that is characteristic of supervised learning-based methods such as HG-Dagger and IWR.

ReIL: A Framework for Reinforced Intervention-based Imitation Learning · Around

Built on

D. A. Pomerleau, “ALVINN: an autonomous land vehicle in a neural network,” in Advances in Neural Information Processing Systems 1 , D. S. Touretzky, Ed. San Francisco, CA: Morgan Kaufmann, 1989, pp. 305–313
1989
Earlier work this paper cites.
S. Ross, G. Gordon, and D. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , ser. Proceedings of Machine Learning Research, G. Gordon, D. Dunson, and M. Dudík, Eds., vol. 15. Fort Lauderdale, FL, USA: PMLR, 11–13 Apr 2011, pp. 627–635. [Online]. Available: https://proceedings.mlr.press/v15/ross11a.html
2011
Earlier work this paper cites.
A. Mandlekar, D. Xu, R. Martín-Martín, Y. Zhu, L. Fei-Fei, and S. Savarese, “Human-in-the-loop imitation learning using remote teleoperation,” CoRR , vol. abs/2012.06733, 2020. [Online]. Available: https://arxiv.org/abs/2012.06733
Original
2012
Earlier work this paper cites.
A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior, and K. Kavukcuoglu, “Wavenet: A generative model for raw audio,” in Arxiv , 2016. [Online]. Available: https://arxiv.org/abs/1609.03499
Original
2016
Earlier work this paper cites.
G. Huang, Z. Liu, and K. Q. Weinberger, “Densely connected convolutional networks,” CoRR , vol. abs/1608.06993, 2016. [Online]. Available: http://arxiv.org/abs/1608.06993
Original
2016
Earlier work this paper cites.
M. Laskey, J. Lee, R. Fox, A. Dragan, and K. Goldberg, “Dart: Noise injection for robust imitation learning,” in Proceedings of the 1st Annual Conference on Robot Learning , ser. Proceedings of Machine Learning Research, S. Levine, V. Vanhoucke, and K. Goldberg, Eds., vol. 78. PMLR, 13–15 Nov 2017, pp. 143–156. [Online]. Available: https://proceedings.mlr.press/v78/laskey17a.html
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017
2017
Earlier work this paper cites.

Similar

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” CoRR , vol. abs/1707.06347, 2017. [Online]. Available: http://arxiv.org/abs/1707.06347
Original
2017
Cited alongside, same era.
X. B. Peng, P. Abbeel, S. Levine, and M. van de Panne, “Deepmimic,” ACM Transactions on Graphics , vol. 37, no. 4, p. 1–14, Aug 2018. [Online]. Available: http://dx.doi.org/10.1145/3197517.3201311
2018
Cited alongside, same era.
S. Fujimoto, H. van Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” CoRR , vol. abs/1802.09477, 2018. [Online]. Available: http://arxiv.org/abs/1802.09477
Original
2018
Cited alongside, same era.
M. Kelly, C. Sidrane, K. R. Driggs-Campbell, and M. J. Kochenderfer, “Hg-dagger: Interactive imitation learning with human experts,” CoRR , vol. abs/1810.02890, 2018. [Online]. Available: http://arxiv.org/abs/1810.02890
Original
2018
Cited alongside, same era.

Then

A. Kendall, J. Hawke, D. Janz, P. Mazur, D. Reda, J.-M. Allen, V.-D. Lam, A. Bewley, and A. Shah, “Learning to drive in a day,” 2019 International Conference on Robotics and Automation (ICRA) , pp. 8248–8254, 2019
2019
Later among the works it cites.
J. Schrittwieser, I. Antonoglou, T. Hubert, K. Simonyan, L. Sifre, S. Schmitt, A. Guez, E. Lockhart, D. Hassabis, T. Graepel, and et al., “Mastering atari, go, chess and shogi by planning with a learned model,” Nature , vol. 588, no. 7839, p. 604–609, Dec 2020. [Online]. Available: http://dx.doi.org/10.1038/s41586-020-03051-4
2020
Later among the works it cites.
J. Spencer, S. Choudhury, M. Barnes, M. Schmittle, M. Chiang, P. J. Ramadge, and S. S. Srinivasa, “Learning from interventions: Human-robot interaction as both explicit and implicit feedback,” Robotics: Science and Systems XVI , 2020
2020
Later among the works it cites.
Z. Peng, Q. Li, C. Liu, and B. Zhou, “Safe driving via expert guided policy optimization,” CoRR , vol. abs/2110.06831, 2021. [Online]. Available: https://arxiv.org/abs/2110.06831

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

F. Wang, B. Zhou, K. Chen, T. Fan, X. Zhang, J. Li, H. Tian, and J. Pan, “Intervention aided reinforcement learning for safe and practical policy optimization in navigation,” in Proceedings of The 2nd Conference on Robot Learning , ser. Proceedings of Machine Learning Research, A. Billard, A. Dragan, J. Peters, and J. Morimoto, Eds., vol. 87. PMLR, 29–31 Oct 2018, pp. 410–421. [Online]. Available: https://proceedings.mlr.press/v87/wang18a.html

2018

Cited alongside, same era.

N. Mishra, M. Rohaninejad, X. Chen, and P. Abbeel, “A simple neural attentive meta-learner,” in 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings , 2018. [Online]. Available: https://openreview.net/forum?id=B1DmUzWAW

2018

Cited alongside, same era.

2021

Later among the works it cites.

S. Fujimoto and S. S. Gu, “A minimalist approach to offline reinforcement learning,” CoRR , vol. abs/2106.06860, 2021. [Online]. Available: https://arxiv.org/abs/2106.06860

2021

Later among the works it cites.

O. Press, N. A. Smith, and M. Lewis, “Train short, test long: Attention with linear biases enables input length extrapolation,” CoRR , vol. abs/2108.12409, 2021. [Online]. Available: https://arxiv.org/abs/2108.12409

2021

Later among the works it cites.