Self-Aligning Depth-regularized Radiance Fields for Asynchronous RGB-D Sequences

Understand

It has been shown that learning radiance fields with depth rendering and depth supervision can effectively promote the quality and convergence of view synthesis.

• However, this paradigm requires input RGB-D sequences to be synchronized, hindering its usage in the UAV city modeling scenario.
• As there exists asynchrony between RGB images and depth images due to high-speed flight, we propose a novel time-pose function, which is an implicit network that maps timestamps to $\rm SE(3)$ elements.
• To simplify the training process, we also design a joint optimization scheme to jointly learn the large-scale depth-regularized radiance fields and the time-pose function.