Fighting Failures with FIRE: Failure Identification to Reduce Expert Burden in Intervention-Based Learning

Understand

Supervised imitation learning, also known as behavioral cloning, suffers from distribution drift leading to failures during policy execution.

• One approach to mitigate this issue is to allow an expert to correct the agent's actions during task execution, based on the expert's determination that the agent has reached a `point of no return.' The agent's policy is then retrained using this new corrective data.
• This approach alone can enable high-performance agents to be learned, but at a substantial cost: the expert must vigilantly observe execution until the policy reaches a specified level of success, and even at that point, there is no guarantee that the policy will always succeed.
• To address these limitations, we present FIRE (Failure Identification to Reduce Expert Burden in intervention-based learning), a system that can predict when a running policy will fail, halt its execution, and request a correction from the expert.