Towards Shutdownable Agents via Stochastic Choice

Understand

The POST-Agents Proposal (PAP) is an idea for ensuring that advanced artificial agents never resist shutdown.

• A key part of the PAP is using a novel `Discounted Reward for Same-Length Trajectories (DReST)' reward function to train agents to (1) pursue goals effectively conditional on each trajectory-length (be `USEFUL'), and (2) choose stochastically between different trajectory-lengths (be `NEUTRAL' about trajectory-lengths).
• In this paper, we propose evaluation metrics for USEFULNESS and NEUTRALITY.
• We use a DReST reward function to train simple agents to navigate gridworlds, and we find that these agents learn to be USEFUL and NEUTRAL.