Multigrid Predictive Filter Flow for Unsupervised Learning on Videos

Understand

We introduce multigrid Predictive Filter Flow (mgPFF), a framework for unsupervised learning on videos.

• The mgPFF takes as input a pair of frames and outputs per-pixel filters to warp one frame to the other.
• Compared to optical flow used for warping frames, mgPFF is more powerful in modeling sub-pixel movement and dealing with corruption (e.g., motion blur).
• We develop a multigrid coarse-to-fine modeling strategy that avoids the requirement of learning large filters to capture large displacement.