Value Memory Graph: A Graph-Structured World Model for Offline Reinforcement Learning

Understand

Reinforcement Learning (RL) methods are typically applied directly in environments to learn policies.

• In some complex environments with continuous state-action spaces, sparse rewards, and/or long temporal horizons, learning a good policy in the original environments can be difficult.
• Focusing on the offline RL setting, we aim to build a simple and discrete world model that abstracts the original environment.
• RL methods are applied to our world model instead of the environment data for simplified policy learning.