Fetching the paper…
Reading the bibliography…
Large language models (LLMs) are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation in the form of reasoning and planning.
Procedures as a representation for data in a computer program for understanding natural language
T. Winograd · 1971
Earlier work this paper cites.
A unified approach for motion and force control of robot manipulators: The operational space formulation
O. Khatib · 1987
Earlier work this paper cites.
A potential field approach to path planning
Y. K. Hwang, N. Ahuja, et al · 1992
Earlier work this paper cites.
Probabilistic roadmaps for path planning in high-dimensional configuration spaces
L. E. Kavraki, P. Svestka, J.-C. Latombe, and M. H. Overmars · 1996
Earlier work this paper cites.
Language conditioned imitation learning over unstructured data
C. Lynch and P. Sermanet · 2005
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Toward understanding natural language directions
T. Kollar, S. Tellex, D. Roy, and N. Roy · 2010
Earlier work this paper cites.
Understanding natural language commands for robotic navigation and mobile manipulation
S. Tellex, T. Kollar, S. Dickerson, M. Walter, A. Banerjee, S. Teller, and N. Roy · 2011
Earlier work this paper cites.
Mujoco: A physics engine for model-based control
E. Todorov, T. Erez, and Y. Tassa · 2012
Earlier work this paper cites.
Interpreting and executing recipes with a cooking robot
M. Bollini, S. Tellex, T. Thompson, N. Roy, and D. Rus · 2013
Earlier work this paper cites.
Asking for help using inverse semantics
S. Tellex, R. Knepper, A. Li, D. Rus, and N. Roy · 2014
Earlier work this paper cites.
Grounding verbs of motion in natural language commands to robots
T. Kollar, S. Tellex, D. Roy, and N. Roy · 2014
Earlier work this paper cites.
Learning to interpret natural language commands through human-robot dialog
J. Thomason, S. Zhang, R. J. Mooney, and P. Stone · 2015
Earlier work this paper cites.
Deepmpc: Learning deep latent features for model predictive control
I. Lenz, R. A. Knepper, and A. Saxena · 2015
Earlier work this paper cites.
U-net: Convolutional networks for biomedical image segmentation
O. Ronneberger, P. Fischer, and T. Brox · 2015
Earlier work this paper cites.
A compositional object-based approach to learning physical dynamics
M. B. Chang, T. Ullman, A. Torralba, and J. B. Tenenbaum · 2016
Earlier work this paper cites.
Interaction networks for learning about objects, relations and physics
P. Battaglia, R. Pascanu, M. Lai, D. Jimenez Rezende, et al · 2016
Earlier work this paper cites.
Guided cost learning: Deep inverse optimal control via policy optimization
C. Finn, S. Levine, and P. Abbeel · 2016
Earlier work this paper cites.
3d u-net: learning dense volumetric segmentation from sparse annotation
Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger · 2016
Earlier work this paper cites.
J. Andreas, D. Klein, and S. Levine · 2017
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2017
Earlier work this paper cites.
Modular multitask reinforcement learning with policy sketches
J. Andreas, D. Klein, and S. Levine · 2017
Earlier work this paper cites.
Se3-nets: Learning rigid body motion using deep neural networks
A. Byravan and D. Fox · 2017
Earlier work this paper cites.
Learning robust rewards with adversarial inverse reinforcement learning
J. Fu, K. Luo, and S. Levine · 2017
Earlier work this paper cites.
Graph networks as learnable physics engines for inference and control
A. Sanchez-Gonzalez, N. Heess, J. T. Springenberg, J. Merel, M. Riedmiller, R. Hadsell, and P. Battaglia · 2018
Earlier work this paper cites.
Learning particle dynamics for manipulating rigid bodies, deformable objects, and fluids
Y. Li, J. Wu, R. Tedrake, J. B. Tenenbaum, and A. Torralba · 2018
Earlier work this paper cites.
Differentiable mpc for end-to-end planning and control
B. Amos, I. Jimenez, J. Sacks, B. Boots, and J. Z. Kolter · 2018
Earlier work this paper cites.
Scaling egocentric vision: The epic-kitchens dataset
D. Damen, H. Doughty, G. M. Farinella, S. Fidler, A. Furnari, E. Kazakos, D. Moltisanti, J. Munro, T. Perrett, W. Price, et al · 2018
Earlier work this paper cites.
N. D. Ratliff, J. Issac, D. Kappler, S. Birchfield, and D. Fox · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever · 2019
Earlier work this paper cites.
A survey of reinforcement learning informed by natural language
J. Luketina, N. Nardelli, G. Farquhar, J. N. Foerster, J. Andreas, E. Grefenstette, S. Whiteson, and T. Rocktäschel · 2019
Earlier work this paper cites.
Language as an abstraction for hierarchical deep reinforcement learning
Y. Jiang, S. S. Gu, K. P. Murphy, and C. Finn · 2019
Earlier work this paper cites.
Densephysnet: Learning dense physical object representations via multi-step dynamic interactions
Z. Xu, J. Wu, A. Zeng, J. B. Tenenbaum, and S. Song · 2019
Earlier work this paper cites.
Language models are few-shot learners
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al · 2020
Earlier work this paper cites.
Robots that use language
S. Tellex, N. Gopalan, H. Kress-Gazit, and C. Matuszek · 2020
Earlier work this paper cites.
Array programming with NumPy
C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del Río, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant · 2020
Earlier work this paper cites.
Unsupervised commonsense question answering with self-talk
V. Shwartz, P. West, R. L. Bras, C. Bhagavatula, and Y. Choi · 2020
Earlier work this paper cites.
Few-shot object grounding and mapping for natural language robot instruction following
V. Blukis, R. A. Knepper, and Y. Artzi · 2020
Earlier work this paper cites.
Jointly improving parsing and perception for natural language commands through human-robot dialog
J. Thomason, A. Padmakumar, J. Sinapov, N. Walker, Y. Jiang, H. Yedidsion, J. Hart, P. Stone, and R. Mooney · 2020
Earlier work this paper cites.
P. A. Jansen · 2020
Earlier work this paper cites.
Language as a cognitive tool to imagine goals in curiosity driven exploration
C. Colas, T. Karch, N. Lair, J.-M. Dussoux, C. Moulin-Frier, P. Dominey, and P.-Y. Oudeyer · 2020
Earlier work this paper cites.
Learning-based model predictive control: Toward safe learning in control
L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger · 2020
Earlier work this paper cites.
Deep dynamics models for learning dexterous manipulation
A. Nagabandi, K. Konolige, S. Levine, and V. Kumar · 2020
Earlier work this paper cites.
Deep visual heuristics: Learning feasibility of mixed-integer programs for manipulation planning
D. Driess, O. Oguz, J.-S. Ha, and M. Toussaint · 2020
Earlier work this paper cites.
Sapien: A simulated part-based interactive environment
F. Xiang, Y. Qin, K. Mo, Y. Xia, H. Zhu, F. Liu, M. Liu, H. Jiang, Y. Yuan, H. Wang, et al · 2020
Cited alongside, same era.
Rlbench: The robot learning benchmark & learning environment
S. James, Z. Ma, D. R. Arrojo, and A. J. Davison · 2020
Cited alongside, same era.
On the opportunities and risks of foundation models
R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al · 2021
Cited alongside, same era.
Learning transferable visual models from natural language supervision
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al · 2021
Cited alongside, same era.
Open-vocabulary object detection via vision and language knowledge distillation
X. Gu, T.-Y. Lin, W. Kuo, and Y. Cui · 2021
Cited alongside, same era.
Training language models to follow instructions with human feedback
L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al · 2022
Later among the works it cites.
Constitutional ai: Harmlessness from ai feedback
Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al · 2022
Later among the works it cites.
Chain of thought prompting elicits reasoning in large language models
J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou · 2022
Later among the works it cites.
Self-instruct: Aligning language model with self generated instructions
Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Mdetr-modulated detection for end-to-end multi-modal understanding
A. Kamath, M. Singh, Y. LeCun, G. Synnaeve, I. Misra, and N. Carion · 2021
Cited alongside, same era.
Bc-z: Zero-shot task generalization with robotic imitation learning
E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn · 2021
Cited alongside, same era.
Language models are few-shot butlers
V. Micheli and F. Fleuret · 2021
Cited alongside, same era.
Skill induction and planning with latent language
P. Sharma, A. Torralba, and J. Andreas · 2021
Cited alongside, same era.
Concept2robot: Learning manipulation concepts from instructions and human demonstrations
L. Shao, T. Migimatsu, Q. Zhang, K. Yang, and J. Bohg · 2021
Cited alongside, same era.
Where2act: From pixels to actions for articulated 3d objects
K. Mo, L. J. Guibas, M. Mukadam, A. Gupta, and S. Tulsiani · 2021
Cited alongside, same era.
Palm: Scaling language modeling with pathways
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al · 2022
Cited alongside, same era.
T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa · 2022
Later among the works it cites.
Emergent abilities of large language models
J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al · 2022
Later among the works it cites.
Viola: Imitation learning for vision-based manipulation with object proposal priors
Y. Zhu, A. Joshi, P. Stone, and Y. Zhu · 2022
Later among the works it cites.
Gpt-4 technical report
OpenAI · 2023
Closest in time.
” no, to the right”–online language corrections for robotic manipulation via shared autonomy
Y. Cui, S. Karamcheti, R. Palleti, N. Shivakumar, P. Liang, and D. Sadigh · 2023
Closest in time.
Open-world object manipulation using pre-trained vision-language models
A. Stone, T. Xiao, Y. Lu, K. Gopalakrishnan, K.-H. Lee, Q. Vuong, P. Wohlhart, B. Zitkovich, F. Xia, C. Finn, et al · 2023
Closest in time.
Liv: Language-image representations and rewards for robotic control
Y. J. Ma, V. Kumar, A. Zhang, O. Bastani, and D. Jayaraman · 2023
Closest in time.
Lampp: Language models as probabilistic priors for perception and action
B. Z. Li, W. Chen, P. Sharma, and J. Andreas · 2023
Closest in time.
Llm+ p: Empowering large language models with optimal planning proficiency
B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone · 2023
Closest in time.
Chatgpt for robotics: Design principles and model abilities
S. Vemprala, R. Bonatti, A. Bucker, and A. Kapoor · 2023
Closest in time.
Text2motion: From natural language instructions to feasible plans
K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg · 2023
Closest in time.
Task and motion planning with large language models for object rearrangement
Y. Ding, X. Zhang, C. Paxton, and S. Zhang · 2023
Closest in time.
Grounded decoding: Guiding text generation with grounded models for robot control
W. Huang, F. Xia, D. Shah, D. Driess, A. Zeng, Y. Lu, P. Florence, I. Mordatch, S. Levine, K. Hausman, et al · 2023
Closest in time.
Palm-e: An embodied multimodal language model
D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, B. Ichter, A. Wahid, J. Tompson, Q. Vuong, T. Yu, et al · 2023
Closest in time.
Plan4mc: Skill reinforcement learning and planning for open-world minecraft tasks
H. Yuan, C. Zhang, H. Wang, F. Xie, P. Cai, H. Dong, and Z. Lu · 2023
Closest in time.
Translating natural language to planning goals with large-language models
Y. Xie, C. Yu, T. Zhu, J. Bai, Z. Gong, and H. Soh · 2023
Closest in time.
Multimodal procedural planning via dual text-image prompting
Y. Lu, P. Lu, Z. Chen, W. Zhu, X. E. Wang, and W. Y. Wang · 2023
Closest in time.
Pretrained language models as visual planners for human assistance
D. Patel, H. Eghbalzadeh, N. Kamra, M. L. Iuzzolino, U. Jain, and R. Desai · 2023
Closest in time.
Voyager: An open-ended embodied agent with large language models
G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar · 2023
Closest in time.
Pave the way to grasp anything: Transferring foundation models for universal pick-place robots
J. Yang, W. Tan, C. Jin, B. Liu, J. Fu, R. Song, and L. Wang · 2023
Closest in time.
Reward design with language models
M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh · 2023
Closest in time.
Guiding pretraining in reinforcement learning with large language models
Y. Du, O. Watkins, Z. Wang, C. Colas, T. Darrell, P. Abbeel, A. Gupta, and J. Andreas · 2023
Closest in time.
Language instructed reinforcement learning for human-ai coordination
H. Hu and D. Sadigh · 2023
Closest in time.
Language to rewards for robotic skill synthesis
W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al · 2023
Closest in time.
Scaling up and distilling down: Language-guided robot skill acquisition
H. Ha, P. Florence, and S. Song · 2023
Closest in time.
Eureka: Human-level reward design via coding large language models
Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar · 2023
Closest in time.
Text2reward: Automated dense reward function generation for reinforcement learning
T. Xie, S. Zhao, C. H. Wu, Y. Liu, Q. Luo, V. Zhong, Y. Yang, and T. Yu · 2023
Closest in time.
Learning reward for physical skills using large language model
Y. Zeng and Y. Xu · 2023
Closest in time.
Vision-language models are zero-shot reward models for reinforcement learning
J. Rocamonde, V. Montesinos, E. Nava, E. Perez, and D. Lindner · 2023
Closest in time.
Larg, language-based automatic reward and goal generation
J. Perez, D. Proux, C. Roux, and M. Niemaz · 2023
Closest in time.
Affordances from human videos as a versatile representation for robotics
S. Bahl, R. Mendonca, L. Chen, U. Jain, and D. Pathak · 2023
Closest in time.
Mimicplay: Long-horizon imitation learning by watching human play
C. Wang, L. Fan, J. Sun, R. Zhang, L. Fei-Fei, D. Xu, Y. Zhu, and A. Anandkumar · 2023
Closest in time.
Zero-shot robot manipulation from passive human videos
H. Bharadhwaj, A. Gupta, S. Tulsiani, and V. Kumar · 2023
Closest in time.
Scaling robot learning with semantically imagined experience
T. Yu, T. Xiao, A. Stone, J. Tompson, A. Brohan, S. Wang, J. Singh, C. Tan, J. Peralta, B. Ichter, et al · 2023
Closest in time.
Behavior-1k: A benchmark for embodied ai with 1,000 everyday activities and realistic simulation
C. Li, R. Zhang, J. Wong, C. Gokmen, S. Srivastava, R. Martín-Martín, C. Wang, G. Levine, M. Lingelbach, J. Sun, et al · 2023
Closest in time.
A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, et al · 2023
Closest in time.
J. Li, D. Li, S. Savarese, and S. Hoi · 2023
Closest in time.
Tree of thoughts: Deliberate problem solving with large language models
S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, and K. Narasimhan · 2023
Closest in time.
Visual programming: Compositional visual reasoning without training
T. Gupta and A. Kembhavi · 2023
Closest in time.
Vipergpt: Visual inference via python execution for reasoning
D. Surís, S. Menon, and C. Vondrick · 2023
Closest in time.