Opening the Vocabulary of Egocentric Actions

Understand

Human actions in egocentric videos are often hand-object interactions composed from a verb (performed by the hand) applied to an object.

• Despite their extensive scaling up, egocentric datasets still face two limitations - sparsity of action compositions and a closed set of interacting objects.
• This paper proposes a novel open vocabulary action recognition task.
• Given a set of verbs and objects observed during training, the goal is to generalize the verbs to an open vocabulary of actions with seen and novel objects.