YRC-Bench: A Benchmark for Learning to Coordinate with Experts

Understand

When deployed in the real world, AI agents will inevitably face challenges that exceed their individual capabilities.

• A critical component of AI safety is an agent's ability to recognize when it is likely to fail in a novel situation and to yield control to a more capable expert system.
• Leveraging such expert assistance can significantly improve safety and performance in such situations.
• Since expert assistance is costly, a central challenge is determining when to consult an expert.