SpotLight: Robotic Scene Understanding through Interaction and Affordance Detection

Understand

Despite increasing research efforts on household robotics, robots intended for deployment in domestic settings still struggle with more complex tasks such as interacting with functional elements like drawers or light switches, largely due to limited task-specific understanding and interaction capabilities.

• These tasks require not only detection and pose estimation but also an understanding of the affordances these elements provide.
• To address these challenges and enhance robotic scene understanding, we introduce SpotLight: A comprehensive framework for robotic interaction with functional elements, specifically light switches.
• Furthermore, this framework enables robots to improve their environmental understanding through interaction.