Zipfian environments for Reinforcement Learning

Understand

As humans and animals learn in the natural world, they encounter distributions of entities, situations and events that are far from uniform.

• Typically, a relatively small set of experiences are encountered frequently, while many important experiences occur only rarely.
• The highly-skewed, heavy-tailed nature of reality poses particular learning challenges that humans and animals have met by evolving specialised memory systems.
• By contrast, most popular RL environments and benchmarks involve approximately uniform variation of properties, objects, situations or tasks.