Fetching the paper…
Reading the bibliography…
Multi-task ``vision-language-action'' (VLA) models have recently demonstrated increasing promise as generalist foundation models for robotics, achieving non-trivial performance out of the box on new tasks in new environments.
Language models are few-shot learners
T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei · 2005
Earlier work this paper cites.
The surprising effectiveness of representation learning for visual imitation
J. Pari, N. M. Shafiullah, S. P. Arunachalam, and L. Pinto · 2021
Earlier work this paper cites.
S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gimenez, Y. Sulsky, J. Kay, J. T. Springenberg, et al · 2022
Earlier work this paper cites.
Improving language models by retrieving from trillions of tokens
S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. B. Van Den Driessche, J.-B. Lespiau, B. Damoc, A. Clark, et al · 2022
Earlier work this paper cites.
Robocat: A self-improving foundation agent for robotic manipulation
K. Bousmalis, G. Vezzani, D. Rao, C. Devin, A. X. Lee, M. Bauza, T. Davchev, Y. Zhou, A. Gupta, A. Raju, et al · 2023
Earlier work this paper cites.
Rt-2: Vision-language-action models transfer web knowledge to robotic control
B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al · 2023
Earlier work this paper cites.
Open x-embodiment: Robotic learning datasets and rt-x models
A. Padalkar, A. Pooley, A. Jain, A. Bewley, A. Herzog, A. Irpan, A. Khazatsky, A. Rai, A. Singh, A. Brohan, et al · 2023
Earlier work this paper cites.
Retrieval-augmented generation for large language models: A survey
Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, and H. Wang · 2023
Earlier work this paper cites.
Pre-training to learn in context
Y. Gu, L. Dong, F. Wei, and M. Huang · 2023
Earlier work this paper cites.
Finetune like you pretrain: Improved finetuning of zero-shot vision models
S. Goyal, A. Kumar, S. Garg, Z. Kolter, and A. Raghunathan · 2023
Earlier work this paper cites.
Generalization to new sequential decision making tasks with in-context learning
S. C. Raparthy, E. Hambro, R. Kirk, M. Henaff, and R. Raileanu · 2023
Earlier work this paper cites.
Large language models as general pattern machines
S. Mirchandani, F. Xia, P. Florence, B. Ichter, D. Driess, M. G. Arenas, K. Rao, D. Sadigh, and A. Zeng · 2023
Earlier work this paper cites.
Eureka: Human-level reward design via coding large language models
Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar · 2023
Cited alongside, same era.
Dinov2: Learning robust visual features without supervision
M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al · 2023
Cited alongside, same era.
Behavior retrieval: Few-shot imitation learning by querying unlabeled datasets
M. Du, S. Nair, D. Sadigh, and C. Finn · 2023
Cited alongside, same era.
Sigmoid loss for language image pre-training
X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer · 2023
Cited alongside, same era.
π 0 \pi_{0} : A vision-language-action flow model for general robot control
R+ x: Retrieval and execution from everyday human videos
G. Papagiannis, N. Di Palo, P. Vitiello, and E. Johns · 2024
Later among the works it cites.
In-context learning enables robot action prediction in llms
Y. Yin, Z. Wang, Y. Sharma, D. Niu, T. Darrell, and R. Herzig · 2024
Later among the works it cites.
Vision language models are in-context value learners
Y. J. Ma, J. Hejna, C. Fu, D. Shah, J. Liang, Z. Xu, S. Kirmani, P. Xu, D. Driess, T. Xiao, et al · 2024
Later among the works it cites.
Droid: A large-scale in-the-wild robot manipulation dataset
A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y. Chen, K. Ellis, et al · 2024
Later among the works it cites.
Memory-consistent neural networks for imitation learning
K. Sridhar, S. Dutta, D. Jayaraman, J. Weimer, and I. Lee · 2024
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al · 2024
Cited alongside, same era.
Openvla: An open-source vision-language-action model
M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al · 2024
Cited alongside, same era.
Octo: An open-source generalist robot policy
O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al · 2024
Cited alongside, same era.
Jack of all trades, master of some, a multi-purpose transformer agent
Q. Gallouédec, E. Beeching, C. Romac, and E. Dellandréa · 2024
Cited alongside, same era.
Genie: Generative interactive environments
J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, et al · 2024
Cited alongside, same era.
In-context imitation learning via next-token prediction
L. Fu, H. Huang, G. Datta, L. Y. Chen, W. C.-H. Panitch, F. Liu, H. Li, and K. Goldberg · 2024
Cited alongside, same era.
Keypoint action tokens enable in-context imitation learning in robotics
N. Di Palo and E. Johns · 2024
Cited alongside, same era.
Later among the works it cites.
Flowretrieval: Flow-guided data retrieval for few-shot imitation learning
L.-H. Lin, Y. Cui, A. Xie, T. Hua, and D. Sadigh · 2024
Later among the works it cites.
Paligemma: A versatile 3b vlm for transfer
L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Alabdulmohsin, M. Tschannen, E. Bugliarello, et al · 2024
Later among the works it cites.
Gemma: Open models based on gemini research and technology
G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivière, M. S. Kale, J. Love, et al · 2024
Later among the works it cites.
Fast: Efficient action tokenization for vision-language-action models
K. Pertsch, K. Stachowicz, B. Ichter, D. Driess, S. Nair, Q. Vuong, O. Mees, C. Finn, and S. Levine · 2025
Closest in time.
Gemini robotics: Bringing ai into the physical world
G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakrishna, R. Baruch, M. Bauza, M. Blokzijl, et al · 2025
Closest in time.
REGENT: A retrieval-augmented generalist agent that can act in-context in new environments
K. Sridhar, S. Dutta, D. Jayaraman, and I. Lee · 2025
Closest in time.