FreDSNet: Joint Monocular Depth and Semantic Segmentation with Fast Fourier Convolutions

Understand

In this work we present FreDSNet, a deep learning solution which obtains semantic 3D understanding of indoor environments from single panoramas.

• Omnidirectional images reveal task-specific advantages when addressing scene understanding problems due to the 360-degree contextual information about the entire environment they provide.
• However, the inherent characteristics of the omnidirectional images add additional problems to obtain an accurate detection and segmentation of objects or a good depth estimation.
• To overcome these problems, we exploit convolutions in the frequential domain obtaining a wider receptive field in each convolutional layer.