ViewFusion: Learning Composable Diffusion Models for Novel View Synthesis

Understand

Deep learning is providing a wealth of new approaches to the problem of novel view synthesis, from Neural Radiance Field (NeRF) based approaches to end-to-end style architectures.

• Each approach offers specific strengths but also comes with limitations in their applicability.
• This work introduces ViewFusion, an end-to-end generative approach to novel view synthesis with unparalleled flexibility.
• ViewFusion consists in simultaneously applying a diffusion denoising step to any number of input views of a scene, then combining the noise gradients obtained for each view with an (inferred) pixel-weighting mask, ensuring that for each region of the target view only the most informative input views are taken into account.