2020

Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations

Ahmadyan, Adel, Zhang, Liangkai, Wei, Jianing et al.

Understand

3D object detection has recently become popular due to many applications in robotics, augmented reality, autonomy, and image retrieval.

We introduce the Objectron dataset to advance the state of the art in 3D object detection and foster new research and applications, such as 3D object tracking, view synthesis, and improved 3D shape representation.
The dataset contains object-centric short videos with pose annotations for nine categories and includes 4 million annotated images in 14,819 annotated videos.
We also propose a new evaluation metric, 3D Intersection over Union, for 3D object detection.

Built on

The Ecological Approach to Visual Perception
James J Gibson · 1979
Earlier work this paper cites.
Real-Time Collision Detection
Christer Ericson · 2004
Earlier work this paper cites.
3D generic object categorization, localization and pose estimation
Silvio Savarese and Fei-Fei Li · 2007
Earlier work this paper cites.
ImageNet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes
Stefan Hinterstoisser, Vincent Lepetit, Slobodan Ilic, Stefan Holzer, Gary Bradski, Kurt Konolige, and Nassir Navab · 2012
Earlier work this paper cites.
Microsoft COCO: Common Objects in Context
Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C Lawrence Zitnick, and Piotr Dollar · 2014
Earlier work this paper cites.
Beyond PASCAL: A benchmark for 3D object detection in the wild
Y Xiang, R Mottaghi, and S Savarese · 2014
Earlier work this paper cites.
ShapeNet: An Information-Rich 3D Model Repository
Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu · 2015
Earlier work this paper cites.
SSD: Single Shot MultiBox Detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg · 2015
Earlier work this paper cites.
Recovering 6D Object Pose and Predicting Next-Best-View in the Crowd
Andreas Doumanoglou, Rigas Kouskouridas, Sotiris Malassiotis, and Tae-Kyun Kim · 2016
Earlier work this paper cites.
3D Bounding Box Estimation Using Deep Learning and and Geometry
Arsalan Mousavian, Dragomir Anguelov, John Flynn, and Jana Kosecka · 2016
Earlier work this paper cites.

Similar

ObjectNet3D: A Large Scale Database for 3D Object Recognition
Yu Xiang, Wonhui Kim, Wei Chen, Jingwei Ji, Christopher Choy, Hao Su, Roozbeh Mottaghi, Leonidas Guibas, and Silvio Savarese · 2016
Cited alongside, same era.
Yale-CMU-Berkeley dataset for robotic manipulation research
Berk Calli, Arjun Singh, James Bruce, Aaron Walsman, Kurt Konolige, Siddhartha Srinivasa, Pieter Abbeel, and Aaron M Dollar · 2017
Cited alongside, same era.
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Niessner · 2017
Cited alongside, same era.
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, and Kevin Murphy · 2017
Cited alongside, same era.
Scan2CAD: Learning CAD Model Alignment in RGB-D Scans
Armen Avetisyan, Manuel Dahnert, Angela Dai, Manolis Savva, Angel X Chang, and Matthias Niessner · 2018
Cited alongside, same era.
BOP: Benchmark for 6D Object Pose Estimation
Tomas Hodan, Rigas Kouskouridas, Tae-Kyun Kim, Federico Tombari, Kostas Bekris, Bertram Drost, Thibault Groueix, Krzysztof Walas, Vincent Lepetit, Ales Leonardis, Carsten Steger, Frank Michel, Caner Sahin, Carsten Rother, and Jiri Matas · 2018
Cited alongside, same era.
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Xingyuan Sun, Jiajun Wu, Xiuming Zhang, Zhoutong Zhang, Chengkai Zhang, Tianfan Xue, Joshua B Tenenbaum, and William T Freeman · 2018
Cited alongside, same era.
Gibson Env: Real-world perception for embodied agents
Fei Xia, Amir R Zamir, Zhi-Yang He, Alexander Sax, Jitendra Malik, and Silvio Savarese · 2018
Cited alongside, same era.
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel · 2019
Cited alongside, same era.
Multiview Aggregation for Learning Category-Specific Shape Reconstruction
Srinath Sridhar, Davis Rempe, Julien Valentin, Bouaziz Sofien, and Leonidas J Guibas · 2019
Cited alongside, same era.
T-LESS: An RGB-D dataset for 6D pose estimation of texture-less objects
Tomas Hodan, Pavel Haluza, Stepan Obdrzalek, Jiri Matas, Manolis Lourakis, and Xenophon Zabulis
Cited in the paper.
A Dataset for Improved RGBD -based Object Detection and Pose Estimation for Warehouse Pick-and-Place
Colin Rennie, Rahul Shome, Kostas E Bekris, and Alberto F De Souza
Cited in the paper.

Then

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Mingxing Tan and Quoc V Le · 2019
Later among the works it cites.
RIO: 3D Object Instance Re-Localization in Changing Indoor Environments
Johanna Wald, Armen Avetisyan, Nassir Navab, Federico Tombari, and Matthias Niessner · 2019
Later among the works it cites.
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
H Wang, S Sridhar, J Huang, J Valentin, and S Song · 2019
Later among the works it cites.
https://developers.google.com/ar
ARcore · 2020
Closest in time.
https://developer.apple.com/augmented-reality/
ARkit · 2020
Closest in time.
MobilePose: Real-Time Pose Estimation for Unseen Objects with Weak Shape Supervision
Tingbo Hou, Adel Ahmadyan, Liangkai Zhang, and Jianing Wei · 2020
Closest in time.
The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari · 2020
Closest in time.
CosyPose: Consistent multi-view multi-object 6D pose estimation
Yann Labbe, Justin Carpentier, Mathieu Aubry, and Josef Sivic · 2020
Closest in time.
Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
Mike Roberts and Nathan Paczan · 2020
Closest in time.
Object-Centric Multi-View Aggregation
Shubham Tulsiani, Or Litany, Charles R Qi, He Wang, and Leonidas J Guibas · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…