UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Loss

Understand

In the era of end-to-end deep learning, many advances in computer vision are driven by large amounts of labeled data.

• In the optical flow setting, however, obtaining dense per-pixel ground truth for real scenes is difficult and thus such data is rare.
• Therefore, recent end-to-end convolutional networks for optical flow rely on synthetic datasets for supervision, but the domain mismatch between training and test scenarios continues to be a challenge.
• Inspired by classical energy-based optical flow methods, we design an unsupervised loss based on occlusion-aware bidirectional flow estimation and the robust census transform to circumvent the need for ground truth flow.