Feature-Based Aggregation and Deep Reinforcement Learning: A Survey and Some New Implementations

Understand

In this paper we discuss policy iteration methods for approximate solution of a finite-state discounted Markov decision problem, with a focus on feature-based aggregation methods and their connection with deep reinforcement learning schemes.

• We introduce features of the states of the original problem, and we formulate a smaller "aggregate" Markov decision problem, whose states relate to the features.
• We discuss properties and possible implementations of this type of aggregation, including a new approach to approximate policy iteration.
• In this approach the policy improvement operation combines feature-based aggregation with feature construction using deep neural networks or other calculations.