2022

NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation

Yuan, Weihao, Gu, Xiaodong, Dai, Zuozhuo et al.

Understand

Estimating the accurate depth from a single image is challenging since it is inherently ambiguous and ill-posed.

While recent works design increasingly complicated and powerful networks to directly regress the depth map, we take the path of CRFs optimization.
Due to the expensive computation, CRFs are usually performed between neighborhoods rather than the whole graph.
To leverage the potential of fully-connected CRFs, we split the input into windows and perform the FC-CRFs optimization within each window, which reduces the computation complexity and makes FC-CRFs feasible.

Built on

Hmm-based surface reconstruction from single images
Takaaki Nagai, Takumi Naruse, Masaaki Ikehara, and Akira Kurematsu · 2002
Earlier work this paper cites.
High speed obstacle avoidance using monocular vision and reinforcement learning
Jeff Michels, Ashutosh Saxena, and Andrew Y Ng · 2005
Earlier work this paper cites.
Learning depth from single monocular images
Ashutosh Saxena, Sung H Chung, Andrew Y Ng, et al · 2005
Earlier work this paper cites.
Learning depth from single monocular images
Ashutosh Saxena, Sung H Chung, Andrew Y Ng, et al · 2005
Earlier work this paper cites.
Make3d: Learning 3d scene structure from a single still image
Ashutosh Saxena, Min Sun, and Andrew Y Ng · 2008
Earlier work this paper cites.
Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera
Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, et al · 2011
Earlier work this paper cites.
Efficient inference in fully connected crfs with gaussian edge potentials
Philipp Krähenbühl and Vladlen Koltun · 2011
Earlier work this paper cites.
Efficient inference in fully connected crfs with gaussian edge potentials
Philipp Krähenbühl and Vladlen Koltun · 2011
Earlier work this paper cites.
Are we ready for autonomous driving? the kitti vision benchmark suite
Andreas Geiger, Philip Lenz, and Raquel Urtasun · 2012
Earlier work this paper cites.
Indoor segmentation and support inference from rgbd images
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus · 2012
Earlier work this paper cites.
Indoor segmentation and support inference from rgbd images
Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus · 2012
Earlier work this paper cites.
Depth map prediction from a single image using a multi-scale deep network
David Eigen, Christian Puhrsch, and Rob Fergus · 2014
Earlier work this paper cites.
Depth and surface normal estimation from monocular images using regression on deep features and hierarchical crfs
Bo Li, Chunhua Shen, Yuchao Dai, Anton Van Den Hengel, and Mingyi He · 2015
Earlier work this paper cites.
Deep convolutional neural fields for depth estimation from a single image
Fayao Liu, Chunhua Shen, and Guosheng Lin · 2015
Earlier work this paper cites.
A depth estimating method from a single image using foe crf
Xiaoyan Wang, Chunping Hou, Liangzhou Pu, and Yonghong Hou · 2015
Earlier work this paper cites.
A depth estimating method from a single image using foe crf
Xiaoyan Wang, Chunping Hou, Liangzhou Pu, and Yonghong Hou · 2015
Earlier work this paper cites.
Depth estimation with convolutional conditional random field network
Yan Hua and Hu Tian · 2016
Earlier work this paper cites.
Matterport3d: Learning from rgb-d data in indoor environments
Original
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang · 2017
Earlier work this paper cites.

Similar

Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia · 2017
Cited alongside, same era.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille · 2017
Cited alongside, same era.
Pyramid scene parsing network
Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia · 2017
Cited alongside, same era.
Deep ordinal regression network for monocular depth estimation
Huan Fu, Mingming Gong, Chaohui Wang, Kayhan Batmanghelich, and Dacheng Tao · 2018
Cited alongside, same era.
Geonet: Geometric neural network for joint depth and surface normal estimation
Xiaojuan Qi, Renjie Liao, Zhengzhe Liu, Raquel Urtasun, and Jiaya Jia · 2018
Cited alongside, same era.
Monocular depth estimation using multi-scale continuous crfs as sequential deep networks
Elisa Ricci, Wanli Ouyang, Xiaogang Wang, Nicu Sebe, et al · 2018
Cited alongside, same era.
Distortion-aware convolutional filters for dense prediction in panoramic images
Keisuke Tateno, Nassir Navab, and Federico Tombari · 2018
Cited alongside, same era.
Structured attention guided convolutional neural fields for monocular depth estimation
Dan Xu, Wei Wang, Hao Tang, Hong Liu, Nicu Sebe, and Elisa Ricci · 2018
Cited alongside, same era.
Omnidepth: Dense depth estimation for indoors spherical panoramas
Nikolaos Zioulis, Antonis Karakottas, Dimitrios Zarpalas, and Petros Daras · 2018
Cited alongside, same era.
From big to small: Multi-scale local planar guidance for monocular depth estimation
Original
Jin Han Lee, Myung-Kyu Han, Dong Wook Ko, and Il Hong Suh · 2019
Cited alongside, same era.

Then

An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Later among the works it cites.
Guiding monocular depth estimation using depth-attention volume
Lam Huynh, Phong Nguyen-Ha, Jiri Matas, Esa Rahtu, and Janne Heikkilä · 2020
Later among the works it cites.
Self-supervised monocular depth estimation: Solving the dynamic object problem by semantic guidance
Marvin Klingner, Jan-Aike Termöhlen, Jonas Mikolajczyk, and Tim Fingscheidt · 2020
Later among the works it cites.
Bifuse: Monocular 360 depth estimation via bi-projection fusion
Fu-En Wang, Yu-Hsuan Yeh, Min Sun, Wei-Chen Chiu, and Yi-Hsuan Tsai · 2020
Later among the works it cites.
Mfusenet: Robust depth estimation with learned multiscopic fusion
Weihao Yuan, Rui Fan, Michael Yu Wang, and Qifeng Chen · 2020
Later among the works it cites.
Bidirectional attention network for monocular depth estimation
Shubhra Aich, Jean Marie Uwabeza Vianney, Md Amirul Islam, Mannat Kaur, and Bingbing Liu · 2021
Later among the works it cites.
Adabins: Depth estimation using adaptive bins
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka · 2021
Later among the works it cites.
Dro: Deep recurrent optimizer for structure-from-motion
Original
Xiaodong Gu, Weihao Yuan, Zuozhuo Dai, Chengzhou Tang, Siyu Zhu, and Ping Tan · 2021
Later among the works it cites.
Sparse auxiliary networks for unified monocular depth prediction and completion
Vitor Guizilini, Rares Ambrus, Wolfram Burgard, and Adrien Gaidon · 2021
Later among the works it cites.
Unifuse: Unidirectional fusion for 360 panorama depth estimation
Hualie Jiang, Zhe Sheng, Siyu Zhu, Zilong Dong, and Rui Huang · 2021
Later among the works it cites.
Patch-wise attention network for monocular depth estimation
Sihaeng Lee, Janghyeon Lee, Byungju Kim, Eojindl Yi, and Junmo Kim · 2021
Later among the works it cites.
Swin transformer: Hierarchical vision transformer using shifted windows
Original
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo · 2021
Later among the works it cites.
Slicenet: deep dense depth estimation from a single indoor panorama using a slice-based representation
Giovanni Pintore, Marco Agus, Eva Almansa, Jens Schneider, and Enrico Gobbetti · 2021
Later among the works it cites.
Vip-deeplab: Learning visual perception with depth-aware video panoptic segmentation
Siyuan Qiao, Yukun Zhu, Hartwig Adam, Alan Yuille, and Liang-Chieh Chen · 2021
Later among the works it cites.
Vision transformers for dense prediction
René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun · 2021
Later among the works it cites.
Hohonet: 360 indoor holistic understanding with latent horizontal features
Cheng Sun, Min Sun, and Hwann-Tzong Chen · 2021
Later among the works it cites.
Stereo matching by self-supervision of multiscopic vision
Weihao Yuan, Yazhan Zhang, Bingkun Wu, Siyu Zhu, Ping Tan, Michael Yu Wang, and Qifeng Chen · 2021
Later among the works it cites.
Adabins: Depth estimation using adaptive bins
Shariq Farooq Bhat, Ibraheem Alhashim, and Peter Wonka · 2021
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…