Cooperative Holistic Scene Understanding: Unifying 3D Object, Layout, and Camera Pose Estimation

Understand

Holistic 3D indoor scene understanding refers to jointly recovering the i) object bounding boxes, ii) room layout, and iii) camera pose, all in 3D.

• The existing methods either are ineffective or only tackle the problem partially.
• In this paper, we propose an end-to-end model that simultaneously solves all three tasks in real-time given only a single RGB image.
• The essence of the proposed method is to improve the prediction by i) parametrizing the targets (e.g., 3D boxes) instead of directly estimating the targets, and ii) cooperative training across different modules in contrast to training these modules individually.