Fetching the paper…
Reading the bibliography…
We present a framework for building interactive, real-time, natural language-instructable robots in the real world, and we open source related assets (dataset, environment, benchmark, and policies).
T. Winograd, “Understanding natural language,” Cognitive psychology , vol. 3, no. 1, pp. 1–191, 1972
1972
Earlier work this paper cites.
D. A. Pomerleau, “Alvinn: An Autonomous Land Vehicle in a Neural Network,” Carnegie-Mellon University, Tech. Rep., 1989
1989
Earlier work this paper cites.
D. A. Pomerleau, “Efficient Training of Artificial Neural Networks for Autonomous Navigation,” Neural Comput. , vol. 3, 1991
1991
Earlier work this paper cites.
R. Caruana, “Multitask learning,” Machine learning , vol. 28, no. 1, pp. 41–75, 1997
1997
Earlier work this paper cites.
S. Schaal, A. Ijspeert, and A. Billard, “Computational approaches to motor learning by imitation,” Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences , vol. 358, no. 1431, pp. 537–547, 2003
2003
Earlier work this paper cites.
S. Schaal, J. Peters, J. Nakanishi, and A. Ijspeert, “Learning movement primitives,” in Robotics research. the eleventh international symposium . Springer, 2005, pp. 561–572
2005
Earlier work this paper cites.
S. R. Branavan, H. Chen, L. Zettlemoyer, and R. Barzilay, “Reinforcement learning for mapping instructions to actions,” in Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP , 2009, pp. 82–90
2009
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255
2009
Earlier work this paper cites.
S. Tellex, T. Kollar, S. Dickerson, M. Walter, A. Banerjee, S. Teller, and N. Roy, “Understanding natural language commands for robotic navigation and mobile manipulation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 25, no. 1, 2011, pp. 1507–1514
2011
Earlier work this paper cites.
D. Chen and R. Mooney, “Learning to interpret natural language navigation instructions from observations,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 25, no. 1, 2011, pp. 859–865
2011
Earlier work this paper cites.
S. M. Khansari-Zadeh and A. Billard, “Learning stable nonlinear dynamical systems with gaussian mixture models,” IEEE Transactions on Robotics , vol. 27, no. 5, pp. 943–957, 2011
2011
Earlier work this paper cites.
P. Kormushev, S. Calinon, and D. G. Caldwell, “Imitation learning of positional and force skills demonstrated via kinesthetic teaching and haptic input,” Advanced Robotics , vol. 25, no. 5, pp. 581–603, 2011
2011
Earlier work this paper cites.
C. Matuszek, E. Herbst, L. Zettlemoyer, and D. Fox, “Learning to parse natural language commands to a robot control system,” in Experimental robotics . Springer, 2013, pp. 403–415
2013
Earlier work this paper cites.
K.-A. Kwon, R. J. Shipley, M. Edirisinghe, D. G. Ezra, G. Rose, S. M. Best, and R. E. Cameron, “High-speed camera characterization of voluntary eye blinking kinematics,” Journal of the Royal Society Interface , vol. 10, no. 85, p. 20130227, 2013
2013
Earlier work this paper cites.
B. Hayes and B. Scassellati, “Challenges in shared-environment human-robot collaboration,” learning , vol. 8, no. 9, 2013
2013
Earlier work this paper cites.
S. Tellex, R. Knepper, A. Li, D. Rus, and N. Roy, “Asking for help using inverse semantics,” 2014
2014
Earlier work this paper cites.
2014
Earlier work this paper cites.
S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” The Journal of Machine Learning Research , vol. 17, no. 1, pp. 1334–1373, 2016
2016
Earlier work this paper cites.
A. Broad, J. Arkin, N. Ratliff, T. Howard, B. Argall, and D. C. Graph, “Towards real-time natural language corrections for assistive robots,” in RSS Workshop on Model Learning for Human-Robot Communication , 2016
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
Earlier work this paper cites.
Y. Bisk, D. Yuret, and D. Marcu, “Natural language communication with robots,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2016, pp. 751–761
2016
Earlier work this paper cites.
E. Coumans and Y. Bai, “Pybullet, a python module for physics simulation for games, robotics and machine learning,” GitHub Repository , 2016
2016
Cited alongside, same era.
2017
Cited alongside, same era.
P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems , vol. 30, 2017
2017
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Cited alongside, same era.
C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, and P. Sermanet, “Learning latent plans from play,” in Conference on robot learning . PMLR, 2020, pp. 1113–1132
2020
Later among the works it cites.
J. Spencer, S. Choudhury, M. Barnes, M. Schmittle, M. Chiang, P. Ramadge, and S. Srinivasa, “Learning from interventions: Human-robot interaction as both explicit and implicit feedback,” in 16th Robotics: Science and Systems, RSS 2020 . MIT Press Journals, 2020
2020
Later among the works it cites.
R. Xiong, Y. Yang, D. He, K. Zheng, S. Zheng, C. Xing, H. Zhang, Y. Lan, L. Wang, and T. Liu, “On layer normalization in the transformer architecture,” in International Conference on Machine Learning . PMLR, 2020, pp. 10 524–10 533
2020
Later among the works it cites.
2021
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, J. Peters et al. , “An algorithmic perspective on imitation learning,” Foundations and Trends® in Robotics , vol. 7, no. 1-2, pp. 1–179, 2018
2018
Cited alongside, same era.
T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,” in 2018 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2018, pp. 5628–5635
2018
Cited alongside, same era.
R. Rahmatizadeh, P. Abolghasemi, L. Bölöni, and S. Levine, “Vision-based multi-task manipulation for inexpensive robots using end-to-end learning from demonstration,” in 2018 IEEE international conference on robotics and automation (ICRA) . IEEE, 2018, pp. 3758–3765
2018
Cited alongside, same era.
D. Rakita, B. Mutlu, and M. Gleicher, “An autonomous dynamic camera method for effective remote teleoperation,” in 2018 13th ACM/IEEE International Conference on Human-Robot Interaction (HRI) . IEEE, 2018, pp. 325–333
2018
Cited alongside, same era.
2018
Cited alongside, same era.
T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation,” in IEEE International Conference on Robotics and Automation (ICRA) , 2018
2018
Cited alongside, same era.
Y. Bisk, K. J. Shih, Y. Choi, and D. Marcu, “Learning interpretable spatial operations in a rich 3d blocks world,” in Thirty-Second AAAI Conference on Artificial Intelligence , 2018
2018
Cited alongside, same era.
2018
Cited alongside, same era.
Later among the works it cites.
2021
Later among the works it cites.
2021
Later among the works it cites.
2021
Later among the works it cites.
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning . PMLR, 2021, pp. 8748–8763
2021
Later among the works it cites.
B. Wu, S. Nair, F.-F. Li, and C. Finn, “Example-driven model-based reinforcement learning for solving long-horizon visuomotor tasks,” in 5th Annual Conference on Robot Learning , 2021. [Online]. Available: https://openreview.net/forum?id=_daq0uh6yXr
2021
Later among the works it cites.
M. Shridhar, L. Manuelli, and D. Fox, “Cliport: What and where pathways for robotic manipulation,” in Conference on Robot Learning . PMLR, 2022, pp. 894–906
2022
Closest in time.
S. Nair, E. Mitchell, K. Chen, S. Savarese, C. Finn et al. , “Learning language-conditioned robot behavior from offline data and crowd-sourced annotation,” in Conference on Robot Learning . PMLR, 2022, pp. 1303–1315
2022
Closest in time.
E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn, “Bc-z: Zero-shot task generalization with robotic imitation learning,” in Conference on Robot Learning . PMLR, 2022, pp. 991–1002
2022
Closest in time.
2022
Closest in time.
S. Karamcheti, M. Srivastava, P. Liang, and D. Sadigh, “Lila: Language-informed latent actions,” in Conference on Robot Learning . PMLR, 2022, pp. 1379–1390
2022
Closest in time.
2022
Closest in time.
O. Mees, L. Hermann, E. Rosete-Beas, and W. Burgard, “Calvin: A benchmark for language-conditioned policy learning for long-horizon robot manipulation tasks,” IEEE Robotics and Automation Letters , 2022
2022
Closest in time.
P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson, “Implicit behavioral cloning,” in Conference on Robot Learning . PMLR, 2022, pp. 158–168
2022
Closest in time.
2022
Closest in time.
2022
Closest in time.