Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and Events

Understand

Scene Dynamic Recovery (SDR) by inverting distorted Rolling Shutter (RS) images to an undistorted high frame-rate Global Shutter (GS) video is a severely ill-posed problem due to the missing temporal dynamic information in both RS intra-frame scanlines and inter-frame exposures, particularly when prior knowledge about camera/object motions is unavailable.

• Commonly used artificial assumptions on scenes/motions and data-specific characteristics are prone to producing sub-optimal solutions in real-world scenarios.
• To address this challenge, we propose an event-based SDR network within a self-supervised learning paradigm, i.e., SelfUnroll.
• We leverage the extremely high temporal resolution of event cameras to provide accurate inter/intra-frame dynamic information.