NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video

Understand

We present a novel framework named NeuralRecon for real-time 3D scene reconstruction from a monocular video.

• Unlike previous methods that estimate single-view depth maps separately on each key-frame and fuse them later, we propose to directly reconstruct local surfaces represented as sparse TSDF volumes for each video fragment sequentially by a neural network.
• A learning-based TSDF fusion module based on gated recurrent units is used to guide the network to fuse features from previous fragments.
• This design allows the network to capture local smoothness prior and global shape prior of 3D surfaces when sequentially reconstructing the surfaces, resulting in accurate, coherent, and real-time surface reconstruction.