Sim-to-Real Transfer for Optical Tactile Sensing

Understand

Deep learning and reinforcement learning methods have been shown to enable learning of flexible and complex robot controllers.

• However, the reliance on large amounts of training data often requires data collection to be carried out in simulation, with a number of sim-to-real transfer methods being developed in recent years.
• In this paper, we study these techniques for tactile sensing using the TacTip optical tactile sensor, which consists of a deformable tip with a camera observing the positions of pins inside this tip.
• We designed a model for soft body simulation which was implemented using the Unity physics engine, and trained a neural network to predict the locations and angles of edges when in contact with the sensor.