Discrete Sequential Prediction of Continuous Actions for Deep RL

Understand

It has long been assumed that high dimensional continuous control problems cannot be solved effectively by discretizing individual dimensions of the action space due to the exponentially large number of bins over which policies would have to be learned.

• In this paper, we draw inspiration from the recent success of sequence-to-sequence models for structured prediction problems to develop policies over discretized spaces.
• Central to this method is the realization that complex functions over high dimensional spaces can be modeled by neural networks that predict one dimension at a time.
• Specifically, we show how Q-values and policies over continuous spaces can be modeled using a next step prediction model over discretized dimensions.