2015

SceneNet: Understanding Real World Indoor Scenes With Synthetic Data

Handa, Ankur, Patraucean, Viorica, Badrinarayanan, Vijay et al.

Understand

Scene understanding is a prerequisite to many high level tasks for any automated intelligent machine operating in real world environments.

Recent attempts with supervised learning have shown promise in this direction but also highlighted the need for enormous quantity of supervised data --- performance increases in proportion to the amount of data used.
However, this quickly becomes prohibitive when considering the manual labour needed to collect such data.
In this work, we focus our attention on depth based semantic per-pixel labelling as a scene understanding problem and show the potential of computer graphics to generate virtually unlimited labelled data from synthetic 3D scenes.

Built on

ImageNet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
BlenSor: blender sensor simulation toolbox
M. Gschwandtner, R. Kwitt, A. Uhl, and W. Pree · 2011
Earlier work this paper cites.
Interactive furniture layout using interior design guidelines
P. Merrell, E. Schkufza, Z. Li, M. Agrawala, and V. Koltun · 2011
Earlier work this paper cites.
Make It Home: automatic optimization of furniture arrangement
L.-F. Yu, S. K. Yeung, C.-K. Tang, D. Terzopoulos, T. F. Chan, and S. Osher · 2011
Earlier work this paper cites.
Example-based synthesis of 3d object arrangements
M. Fisher, D. Ritchie, M. Savva, T. Funkhouser, and P. Hanrahan · 2012
Earlier work this paper cites.
Real-Time Camera Tracking: When is High Frame-Rate Best?
A. Handa, R. A. Newcombe, A. Angeli, and A. J. Davison · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. Hinton · 2012
Earlier work this paper cites.
Indoor segmentation and support inference from RGBD images
N. Silberman, D. Hoiem, P. Kohli, and R. Fergus · 2012
Earlier work this paper cites.
OpenSurfaces: A richly annotated catalog of surface appearance
S. Bell, P. Upchurch, N. Snavely, and K. Bala · 2013
Earlier work this paper cites.
Indoor semantic segmentation using depth information
C. Couprie, C. Farabet, L. Najman, and Y. LeCun · 2013
Earlier work this paper cites.
Perceptual organization and recognition of indoor scenes from RGB-D images
S. Gupta, P. Arbelaez, and J. Malik · 2013
Earlier work this paper cites.
Sun3d: A database of big spaces reconstructed using sfm and object labels
J. Xiao, A. Owens, and A. Torralba · 2013
Earlier work this paper cites.

Similar

Seeing 3D chairs: exemplar part-based 2D-3D alignment using a large dataset of CAD models
M. Aubry, D. Maturana, A. Efros, B. Russell, and J. Sivic · 2014
Cited alongside, same era.
Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition
C. F. Cadieu, H. Hong, D. L. K. Yamins, N. Pinto, D. Ardila, E. A. Solomon, N. J. Majaj, and J. J. DiCarlo · 2014
Cited alongside, same era.
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
S. Gupta, R. Girshick, P. Arbelaez, and J. Malik · 2014
Cited alongside, same era.
A Benchmark for RGB-D Visual Odometry, 3D Reconstruction and SLAM
A. Handa, T. Whelan, J. B. McDonald, and A. J. Davison · 2014
Cited alongside, same era.
Dense 3d semantic mapping of indoor scenes from rgb-d images
A. Hermans, G. Floros, and B. Leibe · 2014
Cited alongside, same era.
Very deep convolutional networks for large-scale image recognition
Original
K. Simonyan and A. Zisserman · 2014
Cited alongside, same era.
Visualizing and understanding convolutional networks
M. D. Zeiler and R. Fergus · 2014
Cited alongside, same era.
Understanding deep features with computer-generated imagery
M. Aubry and B. Russell · 2015
Cited alongside, same era.
SegNet: A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling
Original
V. Badrinarayanan, A. Handa, and R. Cipolla · 2015
Cited alongside, same era.
Semantic image segmentation with deep convolutional nets and fully connected crfs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2015
Cited alongside, same era.
Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture
D. Eigen and R. Fergus · 2015
Cited alongside, same era.
FlowNet: Learning Optical Flow with Convolutional Networks
P. Fischer, A. Dosovitskiy, E. Ilg, P. Häusser, C. Hazırbaş, V. Golkov, P. van der Smagt, D. Cremers, and T. Brox · 2015
Cited alongside, same era.

Then

Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Original
Y. Gal and Z. Ghahramani · 2015
Closest in time.
Aligning 3D models to RGB-D images of cluttered scenes
S. Gupta, P. A. Arbeláez, R. B. Girshick, and J. Malik · 2015
Closest in time.
Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding
Original
A. Kendall, V. Badrinarayanan, , and R. Cipolla · 2015
Closest in time.
Fully convolutional networks for semantic segmentation
J. Long, E. Shelhamer, and T. Darrell · 2015
Closest in time.
Human-level control through deep reinforcement learning
V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al · 2015
Closest in time.
Learning deconvolution network for semantic segmentation
Original
H. Noh, S. Hong, and B. Han · 2015
Closest in time.
SUN RGB-D: A RGB-D scene understanding benchmark suite
S. Song, S. P. Lichtenberg, and J. Xiao · 2015
Closest in time.
Multi-view convolutional neural networks for 3d shape recognition
H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller · 2015
Closest in time.
Smartannotator an interactive tool for annotating indoor rgbd images
Y.-S. Wong, H.-K. Chu, and N. J. Mitra · 2015
Closest in time.
Perceiving physical object properties by integrating a physics engine with deep learning
J. Wu, I. Yildirim, W. Freeman, and J. Tenenbaum · 2015
Closest in time.
3D ShapeNets: A Deep Representation for Volumetric Shapes
Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang, and J. Xiao · 2015
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…