Improving Self-Supervised Single View Depth Estimation by Masking Occlusion

Understand

Single view depth estimation models can be trained from video footage using a self-supervised end-to-end approach with view synthesis as the supervisory signal.

• This is achieved with a framework that predicts depth and camera motion, with a loss based on reconstructing a target video frame from temporally adjacent frames.
• In this context, occlusion relates to parts of a scene that can be observed in the target frame but not in a frame used for image reconstruction.
• Since the image reconstruction is based on sampling from the adjacent frame, and occluded areas by definition cannot be sampled, reconstructed occluded areas corrupt to the supervisory signal.