Reinforcement Learning for Learning Rate Control

Understand

Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks.

• It is observed that the models trained by SGD are sensitive to learning rates and good learning rates are problem specific.
• We propose an algorithm to automatically learn learning rates using neural network based actor-critic methods from deep reinforcement learning (RL).In particular, we train a policy network called actor to decide the learning rate at each step during training, and a value network called critic to give feedback about quality of the decision (e.g., the goodness of the learning rate outputted by the actor) that the actor made.
• The introduction of auxiliary actor and critic networks helps the main network achieve better performance.