Swoosh! Rattle! Thump! -- Actions that Sound

Understand

Truly intelligent agents need to capture the interplay of all their senses to build a rich physical understanding of their world.

• In robotics, we have seen tremendous progress in using visual and tactile perception; however, we have often ignored a key sense: sound.
• This is primarily due to the lack of data that captures the interplay of action and sound.
• In this work, we perform the first large-scale study of the interactions between sound and robotic action.