VisualEchoes: Spatial Image Representation Learning through Echolocation

Understand

Several animal species (e.g., bats, dolphins, and whales) and even visually impaired humans have the remarkable ability to perform echolocation: a biological sonar used to perceive spatial layout and locate objects in the world.

• We explore the spatial cues contained in echoes and how they can benefit vision tasks that require spatial reasoning.
• First we capture echo responses in photo-realistic 3D indoor scene environments.
• Then we propose a novel interaction-based representation learning framework that learns useful visual features via echolocation.