Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs

Understand

Aiming to produce reinforcement learning (RL) policies that are human-interpretable and can generalize better to novel scenarios, Trivedi et al.

• (2021) present a method (LEAPS) that first learns a program embedding space to continuously parameterize diverse programs from a pre-generated program dataset, and then searches for a task-solving program in the learned program embedding space when given a task.
• Despite the encouraging results, the program policies that LEAPS can produce are limited by the distribution of the program dataset.
• Furthermore, during searching, LEAPS evaluates each candidate program solely based on its return, failing to precisely reward correct parts of programs and penalize incorrect parts.