FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Models

Understand

Recent works on generalizable NeRFs have shown promising results on novel view synthesis from single or few images.

• However, such models have rarely been applied on other downstream tasks beyond synthesis such as semantic understanding and parsing.
• In this paper, we propose a novel framework named FeatureNeRF to learn generalizable NeRFs by distilling pre-trained vision foundation models (e.g., DINO, Latent Diffusion).
• FeatureNeRF leverages 2D pre-trained foundation models to 3D space via neural rendering, and then extract deep features for 3D query points from NeRF MLPs.