2020

ObjectNet Dataset: Reanalysis and Correction

Borji, Ali

Understand

Recently, Barbu et al introduced a dataset called ObjectNet which includes objects in daily life situations.

They showed a dramatic performance drop of the state of the art object recognition models on this dataset.
Due to the importance and implications of their results regarding generalization ability of deep models, we take a second look at their findings.
We highlight a major problem with their work which is applying object recognizers to the scenes containing multiple objects rather than isolated objects.

Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,”
1998
Earlier work this paper cites.
Y. LeCun, F. J. Huang, and L. Bottou, “Learning methods for generic object recognition with invariance to pose and lighting,” in
2004
Earlier work this paper cites.
A. Krizhevsky, G. Hinton
2009
Earlier work this paper cites.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in
2012
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,”
Original
2014
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in
2014
Earlier work this paper cites.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,”
Original
2014
Earlier work this paper cites.

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,”
2015
Cited alongside, same era.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in
2015
Cited alongside, same era.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in
2015
Cited alongside, same era.
A. Borji, S. Izadi, and L. Itti, “ilab-20m: A large-scale controlled object dataset to investigate deep learning,” in
2016
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in
2016
Cited alongside, same era.
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, “Rethinking the inception architecture for computer vision,” in
2016
Cited alongside, same era.
T. B. Brown, D. Mané, A. Roy, M. Abadi, and J. Gilmer, “Adversarial patch,” 2017
2017
Cited alongside, same era.

B. Singh and L. S. Davis, “An analysis of scale invariance in object detection snip,” in
2018
Later among the works it cites.
A. Azulay and Y. Weiss, “Why do deep convolutional networks generalize so poorly to small image transformations?”
2019
Later among the works it cites.
B. Recht, R. Roelofs, L. Schmidt, and V. Shankar, “Do imagenet classifiers generalize to imagenet?”
Original
2019
Later among the works it cites.
A. Barbu, D. Mayo, J. Alverio, W. Luo, C. Wang, D. Gutfreund, J. Tenenbaum, and B. Katz, “Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models,” in
2019
Later among the works it cites.
M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, and Q. V. Le, “Mnasnet: Platform-aware neural architecture search for mobile,” in
2019
Later among the works it cites.
A. Borji and S. M. Iranmanesh, “Empirical upper-bound in object detection and more,”
Original
2019
Later among the works it cites.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…