The Optimal Reward Baseline for Gradient-Based Reinforcement Learning

Understand

There exist a number of reinforcement learning algorithms which learnby climbing the gradient of expected reward.

• Their long-runconvergence has been proved, even in partially observableenvironments with non-deterministic actions, and without the need fora system model.
• However, the variance of the gradient estimator hasbeen found to be a significant practical problem.
• Recent approacheshave discounted future rewards, introducing a bias-variance trade-offinto the gradient estimate.