Welfare and Fairness in Multi-objective Reinforcement Learning

Understand

We study fair multi-objective reinforcement learning in which an agent must learn a policy that simultaneously achieves high reward on multiple dimensions of a vector-valued reward.

• Motivated by the fair resource allocation literature, we model this as an expected welfare maximization problem, for some nonlinear fair welfare function of the vector of long-term cumulative rewards.
• One canonical example of such a function is the Nash Social Welfare, or geometric mean, the log transform of which is also known as the Proportional Fairness objective.
• We show that even approximately optimal optimization of the expected Nash Social Welfare is computationally intractable even in the tabular case.