Fetching the paper…

Reading the bibliography…

2024

Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics

Di Palo, Norman, Johns, Edward

Original
PDF
Post on X

Understand

We show that off-the-shelf text-based Transformers, with no additional training, can perform few-shot in-context visual imitation learning, mapping visual observations to action sequences that emulate the demonstrator's behaviour.

We achieve this by transforming visual observations (inputs) and trajectories of actions (outputs) into sequences of tokens that a text-pretrained Transformer (GPT-4 Turbo) can ingest and generate, via a framework we call Keypoint Action Tokens (KAT).
Despite being trained only on language, we show that these Transformers excel at translating tokenised visual keypoint observations into action trajectories, performing on par or better than state-of-the-art imitation learning (diffusion policies) in the low-data regime on a suite of real-world, everyday tasks.
Rather than operating in the language domain as is typical, KAT leverages text-based Transformers to operate in the vision and action domains to learn general patterns in demonstration data for highly efficient imitation learning, indicating promising new avenues for repurposing natural language models for embodied tasks.

Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics · Around

Built on

Language models are few-shot learners, 2020
Original
Tom B. Brown et al · 2005
Earlier work this paper cites.
Action image representation: Learning scalable deep grasping policies with zero real world data, 2020
Original
Mohi Khansari, Daniel Kappler, Jianlan Luo, Jeff Bingham, and Mrinal Kalakrishnan · 2005
Earlier work this paper cites.
Rethinking attention with performers, 2022
Original
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, David Belanger, Lucy Colwell, and Adrian Weller · 2009
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale, 2021
Original
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2010
Earlier work this paper cites.
One-shot imitation learning, 2017
Original
Yan Duan, Marcin Andrychowicz, Bradly C. Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba · 2017
Earlier work this paper cites.
Model-based reinforcement learning via meta-policy optimization, 2018
Original
Ignasi Clavera, Jonas Rothfuss, John Schulman, Yasuhiro Fujita, Tamim Asfour, and Pieter Abbeel · 2018
Earlier work this paper cites.
Task-embedded control networks for few-shot imitation learning
Stephen James, Michael Bloesch, and Andrew J Davison · 2018
Earlier work this paper cites.
Language models are unsupervised multitask learners, 2019
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever · 2019
Earlier work this paper cites.
Emerging properties in self-supervised vision transformers, 2021
Original
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin · 2021
Earlier work this paper cites.
Learning transferable visual models from natural language supervision, 2021
Original
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever · 2021
Earlier work this paper cites.
Deep vit features as dense visual descriptors, 2022
Original
Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel · 2022
Earlier work this paper cites.

Similar

Training compute-optimal large language models, 2022
Original
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre · 2022
Cited alongside, same era.
Language models as zero-shot planners: Extracting actionable knowledge for embodied agents, 2022
Original
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch · 2022
Cited alongside, same era.
R3m: A universal visual representation for robot manipulation, 2022
Original
Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta · 2022
Cited alongside, same era.
In-context learning and induction heads
Original
Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, et al · 2022

Then

Voxposer: Composable 3d value maps for robotic manipulation with language models, 2023
Original
Wenlong Huang, Chen Wang, Ruohan Zhang, Yunzhu Li, Jiajun Wu, and Li Fei-Fei · 2023
Later among the works it cites.
Language models as zero-shot trajectory generators, 2023
Original
Teyun Kwon, Norman Di Palo, and Edward Johns · 2023
Later among the works it cites.
Code as policies: Language model programs for embodied control, 2023
Original
Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, and Andy Zeng · 2023
Later among the works it cites.
Large language models as general pattern machines, 2023
Original
Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, and Andy Zeng · 2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

Diffusion policy: Visuomotor policy learning via action diffusion, 2023

Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song · 2023

Cited alongside, same era.

Open x-embodiment: Robotic learning datasets and rt-x models, 2023

Open X-Embodiment Collaboration · 2023

Cited alongside, same era.

An unbiased look at datasets for visuo-motor pre-training, 2023

Sudeep Dasari, Mohan Kumar Srirama, Unnat Jain, and Abhinav Gupta · 2023

Cited alongside, same era.

Tapir: Tracking any point with per-frame initialization and temporal refinement, 2023

Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman · 2023

Cited alongside, same era.

Palm-e: An embodied multimodal language model, 2023

Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence · 2023

Cited alongside, same era.

Rvt: Robotic view transformer for 3d object manipulation, 2023

Ankit Goyal, Jie Xu, Yijie Guo, Valts Blukis, Yu-Wei Chao, and Dieter Fox · 2023

Cited alongside, same era.

Mamba: Linear-time sequence modeling with selective state spaces, 2023

Albert Gu and Tri Dao · 2023

Cited alongside, same era.

Rt-2: Vision-language-action models transfer web knowledge to robotic control, 2023a

Original

Anthony Brohan et al

Cited in the paper.

Attention is all you need, 2023

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin · 2023

Later among the works it cites.

Robotap: Tracking arbitrary points for few-shot visual imitation, 2023

Mel Vecerik, Carl Doersch, Yi Yang, Todor Davchev, Yusuf Aytar, Guangyao Zhou, Raia Hadsell, Lourdes Agapito, and Jon Scholz · 2023

Later among the works it cites.

Chatgpt for robotics: Design principles and model abilities, 2023

Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor · 2023

Later among the works it cites.

Transformers learn in-context by gradient descent, 2023

Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov · 2023

Later among the works it cites.

Few-shot in-context imitation learning via implicit graph alignment, 2023

Vitalis Vosylius and Edward Johns · 2023

Later among the works it cites.

Language to rewards for robotic skill synthesis, 2023

Wenhao Yu, Nimrod Gileadi, Chuyuan Fu, Sean Kirmani, Kuang-Huei Lee, Montse Gonzalez Arenas, Hao-Tien Lewis Chiang, Tom Erez, Leonard Hasenclever, Jan Humplik, Brian Ichter, Ted Xiao, Peng Xu, Andy Zeng, Tingnan Zhang, Nicolas Heess, Dorsa Sadigh, Jie Tan, Yuval Tassa, and Fei Xia · 2023

Later among the works it cites.

Train offline, test online: A real robot learning benchmark, 2023

Original

Gaoyue Zhou, Victoria Dean, Mohan Kumar Srirama, Aravind Rajeswaran, Jyothish Pari, Kyle Hatch, Aryan Jain, Tianhe Yu, Pieter Abbeel, Lerrel Pinto, Chelsea Finn, and Abhinav Gupta · 2023

Later among the works it cites.