SMiRL: Surprise Minimizing Reinforcement Learning in Unstable Environments

Understand

Every living organism struggles against disruptive environmental forces to carve out and maintain an orderly niche.

• We propose that such a struggle to achieve and preserve order might offer a principle for the emergence of useful behaviors in artificial agents.
• We formalize this idea into an unsupervised reinforcement learning method called surprise minimizing reinforcement learning (SMiRL).
• SMiRL alternates between learning a density model to evaluate the surprise of a stimulus, and improving the policy to seek more predictable stimuli.