2022

Factory: Fast Contact for Robotic Assembly

Narang, Yashraj, Storey, Kier, Akinola, Iretiayo et al.

Understand

Robotic assembly is one of the oldest and most challenging applications of robotics.

In other areas of robotics, such as perception and grasping, simulation has rapidly accelerated research progress, particularly when combined with modern deep learning.
However, accurately, efficiently, and robustly simulating the range of contact-rich interactions in assembly remains a longstanding challenge.
In this work, we present Factory, a set of physics simulation methods and robot learning tools for such applications.

Built on

A unified approach for motion and force control of robot manipulators: The operational space formulation
Oussama Khatib · 1987
Earlier work this paper cites.
Industrial perspective on research needs and opportunities in manufacturing assembly
Louis A. Martin-Vega, Harold K. Brown, Wade H. Shaw, and Thomas J. Sanders · 1995
Earlier work this paper cites.
Mechanics of Robotic Manipulation
Matthew T. Mason · 2001
Earlier work this paper cites.
Impedance and Interaction Control
Neville Hogan and Stephen Buerger · 2004
Earlier work this paper cites.
Fast contact reduction for dynamics simulation
Ádám Moravánszky and Pierre Terdiman · 2004
Earlier work this paper cites.
Mechanical Assemblies: Their Design, Manufacture, and Role in Product Development
Daniel E. Whitney · 2004
Earlier work this paper cites.
A modular haptic rendering algorithm for stable and transparent 6-DOF manipulation
M.A. Otaduy and M.C. Lin · 2006
Earlier work this paper cites.
Finite element analysis and modeling of structure with bolted joints
Jeong Kim, Joo-Cheol Yoon, and Beom-Soo Kang · 2007
Earlier work this paper cites.
Ghosts and machines: regularized variational methods for interactive simulations of multibodies with dry frictional contacts
Claude Lacoursiere · 2007
Earlier work this paper cites.
Introduction to inverse kinematics with Jacobian transpose, pseudoinverse and damped least squares methods
Samuel R. Buss · 2009
Earlier work this paper cites.
Robotics
Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, and Giuseppe Oriolo · 2009
Earlier work this paper cites.
Efficient collision detection for brittle fracture
Loeiz Glondu, Sara C. Schvartzman, Maud Marchal, Georges Dumont, and Miguel A. Otaduy · 2012
Earlier work this paper cites.
MuJoCo: A physics engine for model-based control
Emanuel Todorov, Tom Erez, and Yuval Tassa · 2012
Earlier work this paper cites.
Implicit multibody penalty-based distributed contact
Hongyi Xu, Yili Zhao, and Jernej Barbič · 2014
Earlier work this paper cites.
Robust contact generation for robot simulation with unstructured meshes
Kris Hauser · 2016
Earlier work this paper cites.
Volumetric hierarchical approximate convex decomposition
Khaled Mamou · 2016
Earlier work this paper cites.
Stable simulation of underactuated compliant hands
Alessio Rocchi, Barrett Ames, Zhi Li, and Kris Hauser · 2016
Earlier work this paper cites.
Chrono: An open source multi-physics dynamics engine
Alessandro Tasora, Radu Serban, Hammad Mazhar, Arman Pazouki, Daniel Melanz, Jonathan Fleischmann, Michael Taylor, Hiroyuki Sugiyama, and Dan Negrut · 2016
Earlier work this paper cites.
l ibfranka examples
Franka Emika · 2017
Earlier work this paper cites.
Deep reinforcement learning for high precision assembly tasks
Tadanobu Inoue, Giovanni De Magistris, Asim Munawar, Tsuyoshi Yokoya, and Ryuki Tachibana · 2017
Earlier work this paper cites.
Modern Robotics: Mechanics, Planning, and Control
Kevin M. Lynch and Frank C. Park · 2017
Earlier work this paper cites.
Learning locomotion skills using DeepRL: does the choice of action space matter?
Xue Bin Peng and Michiel van de Panne · 2017
Earlier work this paper cites.
DeepLoco: Dynamic locomotion skills using hierarchical deep reinforcement learning
Xue Bin Peng, Glen Berseth, Kangkang Yin, and Michiel Van De Panne · 2017
Earlier work this paper cites.
Domain randomization for transferring deep neural networks from simulation to the real world
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel · 2017
Earlier work this paper cites.
Blender: A 3d modelling and rendering package
Blender Foundation · 2018
Earlier work this paper cites.
Learning to dress: Synthesizing human dressing motion via deep reinforcement learning
Alexander Clegg, Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk · 2018
Earlier work this paper cites.
Deep reinforcement learning for robotic assembly of mixed deformable and rigid objects
Jianlan Luo, Eugen Solowjow, Chengtao Wen, Juan Aparicio Ojea, and Alice M. Agogino · 2018
Earlier work this paper cites.
RoboTurk: A crowdsourcing platform for robotic skill learning through imitation
Original
Ajay Mandlekar, Yuke Zhu, Animesh Garg, Jonathan Booher, Max Spero, Albert Tung, Julian Gao, John Emmons, Anchit Gupta, Emre Orbay, Silvio Savarese, and Li Fei-Fei · 2018
Earlier work this paper cites.
Robot dynamics lecture notes
Robotic Systems Lab, ETH Zurich · 2018
Earlier work this paper cites.
Can robots assemble an IKEA chair?
Francisco Suárez-Ruiz, Xian Zhou, and Quang-Cuong Pham · 2018
Earlier work this paper cites.
Learning robotic assembly from CAD
Garrett Thomas, Melissa Chien, Aviv Tamar, Juan Aparicio Ojea, and Pieter Abbeel · 2018
Earlier work this paper cites.
Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards
Original
Mel Vecerik, Todd Hester, Jonathan Scholz, Fumin Wang, Olivier Pietquin, Bilal Piot, Nicolas Heess, Thomas Rothörl, Thomas Lampe, and Martin Riedmiller · 2018
Earlier work this paper cites.
Closing the sim-to-real loop: Adapting simulation randomization with real world experience
Yevgen Chebotar, Ankur Handa, Viktor Makoviychuk, Miles Macklin, Jan Issac, Nathan Ratliff, and Dieter Fox · 2019
Earlier work this paper cites.
A learning framework for high precision industrial assembly
Yongxiang Fan, Jieliang Luo, and Masayoshi Tomizuka · 2019
Earlier work this paper cites.
Interlinked SPH pressure solvers for strong fluid-rigid coupling
Christoph Gissler, Andreas Peer, Stefan Band, Jan Bender, and Matthias Teschner · 2019
Earlier work this paper cites.

Similar

On the similarities and differences among contact models in robot simulation
Peter C. Horak and Jeff C. Trinkle · 2019
Cited alongside, same era.
A dataset of daily interactive manipulation
Yongqiang Huang and Yu Sun · 2019
Cited alongside, same era.
A survey of automated threaded fastening
Zhenzhong Jia, Ankit Bhatia, Reuben M. Aronson, David Bourne, and Matthew T. Mason · 2019
Cited alongside, same era.
Data-driven contact clustering for robot simulation
Myungsin Kim, Jaemin Yoon, Dongwon Son, and Dongjun Lee · 2019
Cited alongside, same era.
Reinforcement learning on variable impedance controller for high-precision robotic assembly
Jianlan Luo, Eugen Solowjow, Chengtao Wen, Juan Aparicio Ojea, Alice M. Agogino, Aviv Tamar, and Pieter Abbeel · 2019
Cited alongside, same era.
Dynamic experience replay
Original
Jieliang Luo and Hui Li · 2019
Cited alongside, same era.
Small steps in physics simulation
Miles Macklin, Kier Storey, Michelle Lu, Pierre Terdiman, Nuttapong Chentanez, Stefan Jeschke, and Matthias Müller · 2019
Cited alongside, same era.
Learning ambidextrous robot grasping policies
Jeffrey Mahler, Matthew Matl, Vishal Satish, Michael Danielczuk, Bill DeRose, Stephen McKinley, and Ken Goldberg · 2019
Cited alongside, same era.
Variable impedance control in end-effector space: An action space for reinforcement learning in contact-rich tasks
Original
Roberto Martín-Martín, Michelle A. Lee, Rachel Gardner, Silvio Savarese, Jeannette Bohg, and Animesh Garg · 2019
Cited alongside, same era.
A practical approach to insertion with variable socket position using deep reinforcement learning
Mel Vecerik, Oleg Sushkov, David Barker, Thomas Rothorl, Todd Hester, and Jon Scholz · 2019
Cited alongside, same era.
Robots assembling machines: learning from the World Robot Summit 2018 Assembly Challenge
Felix von Drigalski, Christian Schlette, Martin Rudorfer, Nikolaus Correll, Joshua C. Triyonoputro, Weiwei Wan, Tokuo Tsuji, and Tetsuyou Watanabe · 2019
Cited alongside, same era.
MAT: Multi-fingered adaptive tactile grasping via deep reinforcement learning
Original
Bohan Wu, Iretiayo Akinola, Jacob Varley, and Peter Allen · 2019
Cited alongside, same era.

Then

On the use of simulation in robotics: Opportunities, challenges, and suggestions for moving forward
HeeSun Choi, Cindy Crump, Christian Duriez, Asher Elmquist, Gregory Hager, David Han, Frank Hearl, Jessica Hodgins, Abhinandan Jain, Frederick Leve, Chen Li, Franziska Meier, Dan Negrut, Ludovic Righetti, Alberto Rodriguez, Jie Tan, and Jeff Trinkle · 2021
Later among the works it cites.
Assistive Tele-op: Leveraging transformers to collect robotic task demonstrations
Original
Henry M. Clever, Ankur Handa, Hammad Mazhar, Kevin Parker, Omer Shapira, Qian Wan, Yashraj Narang, Iretiayo Akinola, Maya Cakmak, and Dieter Fox · 2021
Later among the works it cites.
Residual learning from demonstration: Adapting DMPs for contact-rich manipulation
Original
Todor Davchev, Kevin Sebastian Luck, Michael Burke, Franziska Meier, Stefan Schaal, and Subramanian Ramamoorthy · 2021
Later among the works it cites.
ManipulaTHOR: A framework for visual object manipulation
Kiana Ehsani, Winson Han, Alvaro Herrasti, Eli VanderBilt, Luca Weihs, Eric Kolve, Aniruddha Kembhavi, and Roozbeh Mottaghi · 2021
Later among the works it cites.
ACRONYM: A large-scale grasp dataset based on simulation
Clemens Eppner, Arsalan Mousavian, and Dieter Fox · 2021
Later among the works it cites.
Intersection-free rigid body dynamics
Zachary Ferguson, Minchen Li, Teseo Schneider, Francisca Gil-Ureta, Timothy Langlois, Chenfanfu Jiang, Denis Zorin, Danny M Kaufman, and Daniele Panozzo · 2021
Later among the works it cites.
FlingBot: The unreasonable effectiveness of dynamic manipulation for cloth unfolding
Huy Ha and Shuran Song · 2021
Later among the works it cites.
Towards real-world force-sensitive robotic assembly through deep reinforcement learning in simulations
Marius Hebecker, Jens Lambrecht, and Markus Schmitz · 2021
Later among the works it cites.
Data-efficient hierarchical reinforcement learning for robotic assembly control applications
Zhimin Hou, Jiajun Fei, Yuelin Deng, and Jing Xu · 2021
Later among the works it cites.
Trajectory optimization for manipulation of deformable objects: Assembly of belt drive units
Shiyu Jin, Diego Romeres, Arvind Ragunathan, Devesh K. Jha, and Masayoshi Tomizuka · 2021
Later among the works it cites.
Stability-guaranteed reinforcement learning for contact-rich manipulation
Shahbaz Abdul Khader, Hang Yin, Pietro Falco, and Danica Kragic · 2021
Later among the works it cites.
iGibson 2.0: Object-centric simulation for robot learning of everyday household tasks
Original
Chengshu Li, Fei Xia, Roberto Martín-Martín, Michael Lingelbach, Sanjana Srivastava, Bokui Shen, Kent Vainio, Cem Gokmen, Gokul Dharan, Tanish Jain, Andrey Kurenkov, C. Karen Liu, Hyowon Gweon, Jiajun Wu, Li Fei-Fei, and Silvio Savarese · 2021
Later among the works it cites.
Benchmarking off-the-shelf solutions to robotic assembly tasks
Original
Wenzhao Lian, Tim Kelch, Dirk Holz, Adam Norton, and Stefan Schaal · 2021
Later among the works it cites.
The role of physics-based simulators in robotics
C. Karen Liu and Dan Negrut · 2021
Later among the works it cites.
Robust multi-modal policies for industrial assembly via reinforcement learning and demonstrations: A large-scale study
Original
Jianlan Luo, Oleg Sushkov, Rugile Pevceviciute, Wenzhao Lian, Chang Su, Mel Vecerik, Ning Ye, Stefan Schaal, and Jon Scholz · 2021
Later among the works it cites.
A learning approach to robot-agnostic force-guided high precision assembly
Original
Jieliang Luo and Hui Li · 2021
Later among the works it cites.
rl-games, 2021
Denys Makoviichuk and Viktor Makoviychuk · 2021
Later among the works it cites.
Isaac Gym: High performance GPU-based physics simulation for robot learning
Original
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State · 2021
Later among the works it cites.
Sim-to-real for robotic tactile sensing via physics-based simulation and learned latent projections
Yashraj Narang, Balakumar Sundaralingam, Miles Macklin, Arsalan Mousavian, and Dieter Fox · 2021
Later among the works it cites.
Learning to walk in minutes using massively parallel deep reinforcement learning
Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter · 2021
Later among the works it cites.
Taxim: An example-based simulation model for GelSight tactile sensors
Original
Zilin Si and Wenzhen Yuan · 2021
Later among the works it cites.
Habitat 2.0: Training home assistants to rearrange their habitat
Andrew Szot, Alex Clegg, Eric Undersander, Erik Wijmans, Yili Zhao, John Turner, Noah Maestre, Mustafa Mukadam, Devendra Chaplot, Oleksandr Maksymets, Aaron Gokaslan, Vladimir Vondrus, Sameer Dharur, Franziska Meier, Wojciech Galuba, Angel Chang, Zsolt Kira, Vladlen Koltun, Jitendra Malik, Manolis Savva, and Dhruv Batra · 2021
Later among the works it cites.
Learning sequences of manipulation primitives for robotic assembly
Nghia Vuong, Hung Pham, and Quang-Cuong Pham · 2021
Later among the works it cites.
OSCAR: Data-driven operational space control for adaptive and robust robot manipulation
Original
Josiah Wong, Viktor Makoviychuk, Anima Anandkumar, and Yuke Zhu · 2021
Later among the works it cites.
Learning dense rewards for contact-rich manipulation tasks
Zheng Wu, Wenzhao Lian, Vaibhav Unhelkar, Masayoshi Tomizuka, and Stefan Schaal · 2021
Later among the works it cites.
RoboAssembly: Learning generalizable furniture assembly policy in a novel multi-robot contact-rich simulation environment
Original
Mingxin Yu, Lin Shao, Zhehuan Chen, Tianhao Wu, Qingnan Fan, Kaichun Mo, and Hao Dong · 2021
Later among the works it cites.
Interpreting contact interactions to overcome failure in robot assembly tasks
Peter A. Zachares, Michelle A. Lee, Wenzhao Lian, and Jeannette Bohg · 2021
Later among the works it cites.
Learning insertion primitives with discrete-continuous hybrid action space for robotic assembly tasks
Original
Xiang Zhang, Shiyu Jin, Changhao Wang, Xinghao Zhu, and Masayoshi Tomizuka · 2021
Later among the works it cites.
Assembly Magazine
BNP Media · 2022
Closest in time.
https://twitter.com/kennethbodin/status/1252973291217850368 , 2021
Kenneth Bodin · 2022
Closest in time.
Bullet real-time physics simulation
Erwin Coumans · 2022
Closest in time.
DefGraspSim: Physics-based simulation of grasp outcomes for 3D deformable objects
Isabella Huang, Yashraj Narang, Clemens Eppner, Balakumar Sundaralingam, Miles Macklin, Tucker Hermans, and Dieter Fox · 2022
Closest in time.
Fast & reliable micro screw fastening
KUKA · 2022
Closest in time.
NVIDIA Omniverse Platform
NVIDIA · 2022
Closest in time.
NVIDIA PhysX SDK
NVIDIA · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…