Video Inpainting by Jointly Learning Temporal Structure and Spatial Details

Understand

We present a new data-driven video inpainting method for recovering missing regions of video frames.

• A novel deep learning architecture is proposed which contains two sub-networks: a temporal structure inference network and a spatial detail recovering network.
• The temporal structure inference network is built upon a 3D fully convolutional architecture: it only learns to complete a low-resolution video volume given the expensive computational cost of 3D convolution.
• The low resolution result provides temporal guidance to the spatial detail recovering network, which performs image-based inpainting with a 2D fully convolutional network to produce recovered video frames in their original resolution.