Reinforcement Learning by Value Gradients

Understand

The concept of the value-gradient is introduced and developed in the context of reinforcement learning.

• It is shown that by learning the value-gradients exploration or stochastic behaviour is no longer needed to find locally optimal trajectories.
• This is the main motivation for using value-gradients, and it is argued that learning value-gradients is the actual objective of any value-function learning algorithm for control problems.
• It is also argued that learning value-gradients is significantly more efficient than learning just the values, and this argument is supported in experiments by efficiency gains of several orders of magnitude, in several problem domains.