Deformable kernel networks for guided depth map upsampling

Understand

We address the problem of upsampling a low-resolution (LR) depth map using a registered high-resolution (HR) color image of the same scene.

• Previous methods based on convolutional neural networks (CNNs) combine nonlinear activations of spatially-invariant kernels to estimate structural details from LR depth and HR color images, and regress upsampling results directly from the networks.
• In this paper, we revisit the weighted averaging process that has been widely used to transfer structural details from hand-crafted visual features to LR depth maps.
• We instead learn explicitly sparse and spatially-variant kernels for this task.