Metatrace Actor-Critic: Online Step-size Tuning by Meta-gradient Descent for Reinforcement Learning Control

Understand

Reinforcement learning (RL) has had many successes in both "deep" and "shallow" settings.

• In both cases, significant hyperparameter tuning is often required to achieve good performance.
• Furthermore, when nonlinear function approximation is used, non-stationarity in the state representation can lead to learning instability.
• A variety of techniques exist to combat this --- most notably large experience replay buffers or the use of multiple parallel actors.