Learning Finite State Representations of Recurrent Policy Networks

Understand

Recurrent neural networks (RNNs) are an effective representation of control policies for a wide range of reinforcement and imitation learning problems.

• RNN policies, however, are particularly difficult to explain, understand, and analyze due to their use of continuous-valued memory vectors and observation features.
• In this paper, we introduce a new technique, Quantized Bottleneck Insertion, to learn finite representations of these vectors and features.
• The result is a quantized representation of the RNN that can be analyzed to improve our understanding of memory use and general behavior.