On the Practical Consistency of Meta-Reinforcement Learning Algorithms

Understand

Consistency is the theoretical property of a meta learning algorithm that ensures that, under certain assumptions, it can adapt to any task at test time.

• An open question is whether and how theoretical consistency translates into practice, in comparison to inconsistent algorithms.
• In this paper, we empirically investigate this question on a set of representative meta-RL algorithms.
• We find that theoretically consistent algorithms can indeed usually adapt to out-of-distribution (OOD) tasks, while inconsistent ones cannot, although they can still fail in practice for reasons like poor exploration.