Frequency-based Search-control in Dyna

Understand

Model-based reinforcement learning has been empirically demonstrated as a successful strategy to improve sample efficiency.

• In particular, Dyna is an elegant model-based architecture integrating learning and planning that provides huge flexibility of using a model.
• One of the most important components in Dyna is called search-control, which refers to the process of generating state or state-action pairs from which we query the model to acquire simulated experiences.
• Search-control is critical in improving learning efficiency.