MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency

Understand

Masked Modeling (MM) has demonstrated widespread success in various vision challenges, by reconstructing masked visual patches.

• Yet, applying MM for large-scale 3D scenes remains an open problem due to the data sparsity and scene complexity.
• The conventional random masking paradigm used in 2D images often causes a high risk of ambiguity when recovering the masked region of 3D scenes.
• To this end, we propose a novel informative-preserved reconstruction, which explores local statistics to discover and preserve the representative structured points, effectively enhancing the pretext masking task for 3D scene understanding.