Harnessing Distribution Ratio Estimators for Learning Agents with Quality and Diversity

Understand

Quality-Diversity (QD) is a concept from Neuroevolution with some intriguing applications to Reinforcement Learning.

• It facilitates learning a population of agents where each member is optimized to simultaneously accumulate high task-returns and exhibit behavioral diversity compared to other members.
• In this paper, we build on a recent kernel-based method for training a QD policy ensemble with Stein variational gradient descent.
• With kernels based on $f$-divergence between the stationary distributions of policies, we convert the problem to that of efficient estimation of the ratio of these stationary distributions.