Enter the Matrix: Safely Interruptible Autonomous Systems via Virtualization

Understand

Autonomous systems that operate around humans will likely always rely on kill switches that stop their execution and allow them to be remote-controlled for the safety of humans or to prevent damage to the system.

• It is theoretically possible for an autonomous system with sufficient sensor and effector capability that learn online using reinforcement learning to discover that the kill switch deprives it of long-term reward and thus learn to disable the switch or otherwise prevent a human operator from using the switch.
• This is referred to as the big red button problem.
• We present a technique that prevents a reinforcement learning agent from learning to disable the kill switch.