Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Input

Understand

Humans use social context to specify preferences over behaviors, i.e.

• their reward functions.
• Yet, algorithms for inferring reward models from preference data do not take this social learning view into account.
• Inspired by pragmatic human communication, we study how to extract fine-grained data regarding why an example is preferred that is useful for learning more accurate reward models.