Efficient Semantic Video Segmentation with Per-frame Inference

Understand

For semantic segmentation, most existing real-time deep models trained with each frame independently may produce inconsistent results for a video sequence.

• Advanced methods take into considerations the correlations in the video sequence, e.g., by propagating the results to the neighboring frames using optical flow, or extracting the frame representations with other frames, which may lead to inaccurate results or unbalanced latency.
• In this work, we process efficient semantic video segmentation in a per-frame fashion during the inference process.
• Different from previous per-frame models, we explicitly consider the temporal consistency among frames as extra constraints during the training process and embed the temporal consistency into the segmentation network.