Semantically Safe Robot Manipulation: From Semantic Scene Understanding to Motion Safeguards

Understand

Ensuring safe interactions in human-centric environments requires robots to understand and adhere to constraints recognized by humans as "common sense" (e.g., "moving a cup of water above a laptop is unsafe as the water may spill" or "rotating a cup of water is unsafe as it can lead to pouring its content").

• Recent advances in computer vision and machine learning have enabled robots to acquire a semantic understanding of and reason about their operating environments.
• While extensive literature on safe robot decision-making exists, semantic understanding is rarely integrated into these formulations.
• In this work, we propose a semantic safety filter framework to certify robot inputs with respect to semantically defined constraints (e.g., unsafe spatial relationships, behaviors, and poses) and geometrically defined constraints (e.g., environment-collision and self-collision constraints).