2020

3D-FRONT: 3D Furnished Rooms with layOuts and semaNTics

Fu, Huan, Cai, Bowen, Gao, Lin et al.

Understand

We introduce 3D-FRONT (3D Furnished Rooms with layOuts and semaNTics), a new, large-scale, and comprehensive repository of synthetic indoor scenes highlighted by professionally designed layouts and a large number of rooms populated by high-quality textured 3D models with style compatibility.

From layout semantics down to texture details of individual objects, our dataset is freely available to the academic community and beyond.
Currently, 3D-FRONT contains 18,968 rooms diversely furnished by 3D objects, far surpassing all publicly available scene datasets.
In addition, the 13,151 furniture objects all come with high-quality textures.

Built on

Greedy function approximation: a gradient boosting machine
Jerome H Friedman · 2001
Earlier work this paper cites.
Case studies in the use of roc curve analysis for sensor-based estimates in human computer interaction
James Fogarty, Ryan S Baker, and Scott E Hudson · 2005
Earlier work this paper cites.
Semantic labeling of 3d point clouds for indoor scenes
Hema S Koppula, Abhishek Anand, Thorsten Joachims, and Ashutosh Saxena · 2011
Earlier work this paper cites.
Example-based synthesis of 3d object arrangements
Matthew Fisher, Daniel Ritchie, Manolis Savva, Thomas Funkhouser, and Pat Hanrahan · 2012
Earlier work this paper cites.
Indoor segmentation and support inference from rgbd images
Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus · 2012
Earlier work this paper cites.
A benchmark for the evaluation of rgb-d slam systems
Jürgen Sturm, Nikolas Engelhard, Felix Endres, Wolfram Burgard, and Daniel Cremers · 2012
Earlier work this paper cites.
Sun3d: A database of big spaces reconstructed using sfm and object labels
Jianxiong Xiao, Andrew Owens, and Antonio Torralba · 2013
Earlier work this paper cites.
ShapeNet: An information-rich 3d model repository
Original
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al · 2015
Earlier work this paper cites.
Sun rgb-d: A rgb-d scene understanding benchmark suite
Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao · 2015
Earlier work this paper cites.
3d semantic parsing of large-scale indoor spaces
Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese · 2016
Earlier work this paper cites.
Understanding real world indoor scenes with synthetic data
Ankur Handa, Viorica Patraucean, Vijay Badrinarayanan, Simon Stent, and Roberto Cipolla · 2016
Earlier work this paper cites.
Scenenn: A scene meshes dataset with annotations
Binh-Son Hua, Quang-Hieu Pham, Duc Thanh Nguyen, Minh-Khoi Tran, Lap-Fai Yu, and Sai-Kit Yeung · 2016
Earlier work this paper cites.
Semi-supervised classification with graph convolutional networks
Original
Thomas N Kipf and Max Welling · 2016
Earlier work this paper cites.
PiGraphs: Learning Interaction Snapshots from Observations
Manolis Savva, Angel X. Chang, Pat Hanrahan, Matthew Fisher, and Matthias Nießner · 2016
Earlier work this paper cites.
Joint 2d-3d-semantic data for indoor scene understanding
Original
Iro Armeni, Sasha Sax, Amir R Zamir, and Silvio Savarese · 2017
Earlier work this paper cites.

Similar

Matterport3d: Learning from rgb-d data in indoor environments
Original
Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang · 2017
Cited alongside, same era.
Scannet: Richly-annotated 3d reconstructions of indoor scenes
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner · 2017
Cited alongside, same era.
Scenenet rgb-d: Can 5m synthetic images beat generic imagenet pre-training on indoor segmentation?
John McCormac, Ankur Handa, Stefan Leutenegger, and Andrew J Davison · 2017
Cited alongside, same era.
Semantic scene completion from a single depth image
Shuran Song, Fisher Yu, Andy Zeng, Angel X Chang, Manolis Savva, and Thomas Funkhouser · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Physically-based rendering for indoor scene understanding using convolutional neural networks
Yinda Zhang, Shuran Song, Ersin Yumer, Manolis Savva, Joon-Young Lee, Hailin Jin, and Thomas Funkhouser · 2017
Cited alongside, same era.
Learning representations and generative models for 3d point clouds
Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Original
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Cited alongside, same era.
The robotrix: An extremely photorealistic and very-large-scale indoor dataset of sequences with robot trajectories and interactions
Alberto Garcia-Garcia, Pablo Martinez-Gonzalez, Sergiu Oprea, John Alejandro Castro-Vargas, Sergio Orts-Escolano, Jose Garcia-Rodriguez, and Alvaro Jover-Alvarez · 2018
Cited alongside, same era.
Interiornet: Mega-scale multi-sensor photo-realistic indoor scenes dataset
Original
Wenbin Li, Sajad Saeedi, John McCormac, Ronald Clark, Dimos Tzoumanikas, Qing Ye, Yuzhong Huang, Rui Tang, and Stefan Leutenegger · 2018
Cited alongside, same era.
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Cited alongside, same era.
Pix3d: Dataset and methods for single-image 3d shape modeling
Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman · 2018
Cited alongside, same era.

Then

The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang · 2018
Later among the works it cites.
Scan2CAD: Learning cad model alignment in rgb-d scans
Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X. Chang, and Matthias Niessner · 2019
Later among the works it cites.
Context-aware visual compatibility prediction
Guillem Cucurull, Perouz Taslakian, and David Vazquez · 2019
Later among the works it cites.
Grains: Generative recursive autoencoders for indoor scenes
Manyi Li, Akshay Gadi Patil, Kai Xu, Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe Tu, Baoquan Chen, Daniel Cohen-Or, and Hao Zhang · 2019
Later among the works it cites.
Learning to generate textures on 3d meshes
Amit Raj, Cusuh Ham, Connelly Barnes, Vladimir Kim, Jingwan Lu, and James Hays · 2019
Later among the works it cites.
Fast and flexible indoor scene synthesis via deep convolutional generative models
Daniel Ritchie, Kai Wang, and Yu-an Lin · 2019
Later among the works it cites.
The replica dataset: A digital replica of indoor spaces
Original
Julian Straub, Thomas Whelan, Lingni Ma, Yufan Chen, Erik Wijmans, Simon Green, Jakob J Engel, Raul Mur-Artal, Carl Ren, Shobhit Verma, et al · 2019
Later among the works it cites.
Planit: Planning and instantiating indoor scenes with relation graph and spatial prior networks
Kai Wang, Yu-An Lin, Ben Weissmann, Manolis Savva, Angel X Chang, and Daniel Ritchie · 2019
Later among the works it cites.
Structured3d: A large photo-realistic dataset for structured 3d modeling
Original
Jia Zheng, Junfei Zhang, Jing Li, Rui Tang, Shenghua Gao, and Zihan Zhou · 2019
Later among the works it cites.
3d-future: 3d furniture shape with texture
Original
Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang Zhao, Steve Maybank, and Dacheng Tao · 2020
Closest in time.
Tm-net: Deep generative networks for textured meshes
Original
Lin Gao, Tong Wu, Yu-Jie Yuan, Ming-Xian Lin, Yu-Kun Lai, and Hao Zhang · 2020
Closest in time.
Openrooms: An end-to-end open framework for photorealistic indoor scene datasets
Original
Zhengqin Li, Ting-Wei Yu, Shen Sang, Sarah Wang, Sai Bi, Zexiang Xu, Hong-Xing Yu, Kalyan Sunkavalli, Miloš Hašan, Ravi Ramamoorthi, et al · 2020
Closest in time.
GeLaTO: Generative Latent Textured Objects
Ricardo Martin-Brualla, Rohit Pandey, Sofien Bouaziz, Matthew Brown, and Dan B Goldman · 2020
Closest in time.
Hypersim: A photorealistic synthetic dataset for holistic indoor scene understanding, 2020
Mike Roberts and Nathan Paczan · 2020
Closest in time.
Deep generative modeling for scene synthesis via hybrid representations
Zaiwei Zhang, Zhenpei Yang, Chongyang Ma, Linjie Luo, Alexander Huth, Etienne Vouga, and Qixing Huang · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…