D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation

Understand

We address weakly supervised action alignment and segmentation in videos, where only the order of occurring actions is available during training.

• We propose Discriminative Differentiable Dynamic Time Warping (D3TW), the first discriminative model using weak ordering supervision.
• The key technical challenge for discriminative modeling with weak supervision is that the loss function of the ordering supervision is usually formulated using dynamic programming and is thus not differentiable.
• We address this challenge with a continuous relaxation of the min-operator in dynamic programming and extend the alignment loss to be differentiable.