Tactile-Augmented Radiance Fields

Understand

We present a scene representation, which we call a tactile-augmented radiance field (TaRF), that brings vision and touch into a shared 3D space.

• This representation can be used to estimate the visual and tactile signals for a given 3D position within a scene.
• We capture a scene's TaRF from a collection of photos and sparsely sampled touch probes.
• Our approach makes use of two insights: (i) common vision-based touch sensors are built on ordinary cameras and thus can be registered to images using methods from multi-view geometry, and (ii) visually and structurally similar regions of a scene share the same tactile features.