Video Enhancement with Task-Oriented Flow

Understand

Many video enhancement algorithms rely on optical flow to register frames in a video sequence.

• Precise flow estimation is however intractable; and optical flow itself is often a sub-optimal representation for particular video processing tasks.
• In this paper, we propose task-oriented flow (TOFlow), a motion representation learned in a self-supervised, task-specific manner.
• We design a neural network with a trainable motion estimation component and a video processing component, and train them jointly to learn the task-oriented flow.