Time-Efficient Reward Learning via Visually Assisted Cluster Ranking

Understand

One of the most successful paradigms for reward learning uses human feedback in the form of comparisons.

• Although these methods hold promise, human comparison labeling is expensive and time consuming, constituting a major bottleneck to their broader applicability.
• Our insight is that we can greatly improve how effectively human time is used in these approaches by batching comparisons together, rather than having the human label each comparison individually.
• To do so, we leverage data dimensionality-reduction and visualization techniques to provide the human with a interactive GUI displaying the state space, in which the user can label subportions of the state space.