2022

CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning

Mandi, Zhao, Bharadhwaj, Homanga, Moens, Vincent et al.

Understand

Large-scale training have propelled significant progress in various sub-fields of AI such as computer vision and natural language processing.

However, building robot learning systems at a comparable scale remains challenging.
To develop robots that can perform a wide range of skills and adapt to new scenarios, efficient methods for collecting vast and diverse amounts of data on physical robot systems are required, as well as the capability to train high-capacity policies using such datasets.
In this work, we propose a framework for scaling robot learning, with specific focus on multi-task and multi-scene manipulation in kitchen environments, both in simulation and in the real world.

Built on

Image inpainting
Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, C · 2000
Earlier work this paper cites.
A natural policy gradient
Kakade, S. M · 2001
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Embed to control: A locally linear latent dynamics model for control from raw images
Original
Watter, M., Springenberg, J. T., Boedecker, J., and Riedmiller, M · 2015
Earlier work this paper cites.
Progressive neural networks
Original
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R · 2016
Earlier work this paper cites.
(cad)$ˆ2$rl: Real single-image flight without a single real image
Original
Sadeghi, F. and Levine, S · 2016
Earlier work this paper cites.
Stochastic variational video prediction
Original
Babaeizadeh, M., Finn, C., Erhan, D., Campbell, R. H., and Levine, S · 2017
Earlier work this paper cites.
Deep visual foresight for planning robot motion
Finn, C. and Levine, S · 2017
Earlier work this paper cites.
One-shot visual imitation learning via meta-learning
Finn, C., Yu, T., Zhang, T., Abbeel, P., and Levine, S · 2017
Earlier work this paper cites.
Transferring end-to-end visuomotor control from simulation to real world for a multi-stage task
Original
James, S., Davison, A. J., and Johns, E · 2017
Earlier work this paper cites.
Sim-to-real transfer of robotic control with dynamics randomization
Original
Peng, X. B., Andrychowicz, M., Zaremba, W., and Abbeel, P · 2017
Earlier work this paper cites.
Learning to push by grasping: Using multiple tasks for effective learning
Pinto, L. and Gupta, A · 2017
Earlier work this paper cites.
Domain randomization for transferring deep neural networks from simulation to the real world
Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., and Abbeel, P · 2017
Earlier work this paper cites.
Impala: Scalable distributed deep-rl with importance weighted actor-learner architectures
Espeholt, L., Soyer, H., Munos, R., Simonyan, K., Mnih, V., Ward, T., Doron, Y., Firoiu, V., Harley, T., Dunning, I., et al · 2018
Earlier work this paper cites.
Learning latent dynamics for planning from pixels
Original
Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J · 2018
Earlier work this paper cites.

Similar

Roboturk: A crowdsourcing platform for robotic skill learning through imitation
Mandlekar, A., Zhu, Y., Garg, A., Booher, J., Spero, M., Tung, A., Gao, J., Emmons, J., Gupta, A., Orbay, E., et al · 2018
Cited alongside, same era.
Reptile: a scalable metalearning algorithm
Original
Nichol, A. and Schulman, J · 2018
Cited alongside, same era.
Learning by playing solving sparse reward tasks from scratch
Riedmiller, M., Hafner, R., Lampe, T., Neunert, M., Degrave, J., Wiele, T., Mnih, V., Heess, N., and Springenberg, J. T · 2018
Cited alongside, same era.
Deepmind control suite
Original
Tassa, Y., Doron, Y., Muldal, A., Erez, T., Li, Y., Casas, D. d. L., Budden, D., Abdolmaleki, A., Merel, J., Lefrancq, A., et al · 2018
Cited alongside, same era.
Shaping belief states with generative environment models for rl
Original
Gregor, K., Rezende, D. J., Besse, F., Wu, Y., Merzic, H., and Oord, A. v. d · 2019
Cited alongside, same era.
Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning
Original
Gupta, A., Kumar, V., Lynch, C., Levine, S., and Hausman, K · 2019
Cited alongside, same era.
Dream to control: Learning behaviors by latent imagination
Original
Hafner, D., Lillicrap, T., Ba, J., and Norouzi, M · 2019
Cited alongside, same era.
Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks
James, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., and Bousmalis, K · 2019
Cited alongside, same era.
Stochastic latent actor-critic: Deep reinforcement learning with a latent variable model
Original
Lee, A. X., Nagabandi, A., Abbeel, P., and Levine, S · 2019
Cited alongside, same era.
Learning latent plans from play
Original
Lynch, C., Khansari, M., Xiao, T., Kumar, V., Tompson, J., Levine, S., and Sermanet, P · 2019
Cited alongside, same era.
Skew-fit: State-covering self-supervised reinforcement learning
Original
Pong, V. H., Dalal, M., Lin, S., Nair, A., Bahl, S., and Levine, S · 2019
Cited alongside, same era.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I · 2019
Cited alongside, same era.

Then

Visual imitation made easy
Young, S., Gandhi, D., Tulsiani, S., Gupta, A., Abbeel, P., and Pinto, L · 2020
Later among the works it cites.
Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning
Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., and Levine, S · 2020
Later among the works it cites.
Randomized ensembled double q-learning: Learning fast without a model
Original
Chen, X., Wang, C., Zhou, Z., and Ross, K · 2021
Later among the works it cites.
Mt-opt: Continuous multi-task robotic reinforcement learning at scale
Original
Kalashnikov, D., Varley, J., Chebotar, Y., Swanson, B., Jonschkowski, R., Finn, C., Levine, S., and Hausman, K · 2021
Later among the works it cites.
Ego4d: Around the world in 3,000 hours of egocentric video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al · 2022
Closest in time.
Bc-z: Zero-shot task generalization with robotic imitation learning
Jang, E., Irpan, A., Khansari, M., Kappler, D., Ebert, F., Lynch, C., Levine, S., and Finn, C · 2022
Closest in time.
Dall-e-bot: Introducing web-scale diffusion models to robotics
Original
Kapelyukh, I., Vosylius, V., and Johns, E · 2022
Closest in time.
On the effectiveness of fine-tuning versus meta-reinforcement learning
Original
Mandi, Z., Abbeel, P., and James, S · 2022
Closest in time.
R3m: A universal visual representation for robot manipulation
Original
Nair, S., Rajeswaran, A., Kumar, V., Finn, C., and Gupta, A · 2022
Closest in time.
Hierarchical text-conditional image generation with clip latents
Original
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M · 2022
Closest in time.
A generalist agent
Original
Reed, S., Zolna, K., Parisotto, E., Colmenarejo, S. G., Novikov, A., Barth-Maron, G., Gimenez, M., Sulsky, Y., Kay, J., Springenberg, J. T., et al · 2022
Closest in time.
Visual goal-directed meta-imitation learning
Rivera, C. G., Handelman, D. A., Ratto, C. R., Patrone, D., and Paulhamus, B. L · 2022
Closest in time.
Photorealistic text-to-image diffusion models with deep language understanding
Original
Saharia, C., Chan, W., Saxena, S., Li, L., Whang, J., Denton, E., Ghasemipour, S. K. S., Ayan, B. K., Mahdavi, S. S., Lopes, R. G., et al · 2022
Closest in time.
Behavior transformers: Cloning k k modes with one stone
Original
Shafiullah, N. M. M., Cui, Z. J., Altanzaya, A., and Pinto, L · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…