2019

Tencent ML-Images: A Large-Scale Multi-Label Image Database for Visual Representation Learning

Wu, Baoyuan, Chen, Weidong, Fan, Yanbo et al.

Understand

In existing visual representation learning tasks, deep convolutional neural networks (CNNs) are often trained on images annotated with single tags, such as ImageNet.

However, a single tag cannot describe all important contents of one image, and some useful visual information may be wasted during training.
In this work, we propose to train CNNs from images annotated with multiple tags, to enhance the quality of visual representation of the trained CNN model.
To this end, we build a large-scale multi-label image database with 18M images and 11K categories, dubbed Tencent ML-Images.

Built on

B. Wu, Z. Liu, S. Wang, B.-G. Hu, and Q. Ji, “Multi-label learning with missing labels,” in ICPR , 2014, pp. 1964–1968
1968
Earlier work this paper cites.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Cognitive modeling , vol. 5, no. 3, p. 1, 1988
1988
Earlier work this paper cites.
C. Fellbaum, WordNet . Wiley Online Library, 1998
1998
Earlier work this paper cites.
Y. LeCun, L. Bottou, Y. Bengio, P. Haffner et al. , “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998
1998
Earlier work this paper cites.
P. Duygulu, K. Barnard, J. F. de Freitas, and D. A. Forsyth, “Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary,” in ECCV , 2002, pp. 97–112
2002
Earlier work this paper cites.
L. Von Ahn and L. Dabbish, “Labeling images with a computer game,” in Proceedings of the SIGCHI conference on Human factors in computing systems , 2004, pp. 319–326
2004
Earlier work this paper cites.
M. Grubinger, P. Clough, H. Müller, and T. Deselaers, “The iapr tc-12 benchmark: A new evaluation resource for visual information systems,” in International Workshop OntoImage , 2006, pp. 13–23
2006
Earlier work this paper cites.
G. Griffin, A. Holub, and P. Perona, “Caltech-256 object category dataset,” Technical Report 7694, Caltech , 2007
2007
Earlier work this paper cites.
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in CVPR , 2009, pp. 248–255
2009
Earlier work this paper cites.
A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Technical report, University of Toronto , 2009
2009
Earlier work this paper cites.
T.-S. Chua, J. Tang, R. Hong, H. Li, Z. Luo, and Y.-T. Zheng, “Nus-wide: A real-world web image database from national university of singapore,” in CIVR , 2009, pp. 368–375
2009
Earlier work this paper cites.
H. He and E. A. Garcia, “Learning from imbalanced data,” IEEE Transactions on knowledge and data engineering , vol. 21, no. 9, pp. 1263–1284, 2009
2009
Earlier work this paper cites.
P. Patarasuk and X. Yuan, “Bandwidth optimal all-reduce algorithms for clusters of workstations,” Journal of Parallel and Distributed Computing , vol. 69, no. 2, pp. 117–124, 2009
2009
Earlier work this paper cites.
J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, “Sun database: Large-scale scene recognition from abbey to zoo,” in CVPR , 2010, pp. 3485–3492
2010
Earlier work this paper cites.

Similar

M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes (voc) challenge,” International journal of computer vision , vol. 88, no. 2, pp. 303–338, 2010
2010
Cited alongside, same era.
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in NIPS , 2012, pp. 1097–1105
2012
Cited alongside, same era.
M.-L. Zhang and Z.-H. Zhou, “A review on multi-label learning algorithms,” IEEE transactions on knowledge and data engineering , vol. 26, no. 8, pp. 1819–1837, 2013
2013
Cited alongside, same era.
K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, “Return of the devil in the details: Delving deep into convolutional nets,” arXiv preprint arXiv:1405.3531 , 2014
Original
2014
Cited alongside, same era.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in ECCV , 2014, pp. 740–755
2014
Cited alongside, same era.
B. Wu, S. Lyu, and B. Ghanem, “Ml-mg: Multi-label learning with missing labels using a mixed graph,” in ICCV , 2015, pp. 4157–4165
2015
Cited alongside, same era.
B. Wu, S. Lyu, B.-G. Hu, and Q. Ji, “Multi-label learning with missing labels for image annotation and facial action unit recognition,” Pattern Recognition , vol. 48, no. 7, pp. 2279–2289, 2015
2015
Cited alongside, same era.
C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolutions,” in CVPR , 2015, pp. 1–9
2015
Cited alongside, same era.
M. Everingham, S. A. Eslami, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman, “The pascal visual object classes challenge: A retrospective,” International journal of computer vision , vol. 111, no. 1, pp. 98–136, 2015
2015
Cited alongside, same era.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” in NIPS , 2015, pp. 91–99
2015
Cited alongside, same era.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR , 2016, pp. 770–778
2016
Cited alongside, same era.
B. Wu, S. Lyu, and B. Ghanem, “Constrained submodular minimization for missing labels and class imbalance in multi-label learning.” in AAAI , 2016, pp. 2229–2236
2016
Cited alongside, same era.

Then

K. He, X. Zhang, S. Ren, and J. Sun, “Identity mappings in deep residual networks,” in ECCV , 2016, pp. 630–645
2016
Later among the works it cites.
I. Krasin, T. Duerig, N. Alldrin, V. Ferrari, S. Abu-El-Haija, A. Kuznetsova, H. Rom, J. Uijlings, S. Popov, A. Veit, S. Belongie, V. Gomes, A. Gupta, C. Sun, G. Chechik, D. Cai, Z. Feng, D. Narayanan, and K. Murphy, “Openimages: A public dataset for large-scale multi-label and multi-class image classification.” Dataset available from https://github.com/openimages , 2017
2017
Later among the works it cites.
C. Sun, A. Shrivastava, S. Singh, and A. Gupta, “Revisiting unreasonable effectiveness of data in deep learning era,” in ICCV , 2017, pp. 843–852
2017
Later among the works it cites.
NVIDIA, “Nvidia collective communications library (nccl).” https://developer.nvidia.com/nccl , 2017
2017
Later among the works it cites.
W. Li, L. Wang, W. Li, E. Agustsson, and L. Van Gool, “Webvision database: Visual learning and understanding from web data,” arXiv preprint arXiv:1708.02862 , 2017
Original
2017
Later among the works it cites.
B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 6, pp. 1452–1464, 2017
2017
Later among the works it cites.
A. Veit, N. Alldrin, G. Chechik, I. Krasin, A. Gupta, and S. Belongie, “Learning from noisy large-scale datasets with minimal supervision,” in CVPR , 2017, pp. 839–847
2017
Later among the works it cites.
P. Goyal, P. Dollár, R. Girshick, P. Noordhuis, L. Wesolowski, A. Kyrola, A. Tulloch, Y. Jia, and K. He, “Accurate, large minibatch sgd: training imagenet in 1 hour,” arXiv preprint arXiv:1706.02677 , 2017
Original
2017
Later among the works it cites.
J. Redmon and A. Farhadi, “Yolo9000: Better, faster, stronger,” in CVPR , 2017, pp. 6517–6525
2017
Later among the works it cites.
X. Chen and A. Gupta, “Spatial memory for context reasoning in object detection,” in ICCV , 2017, pp. 4086–4096
2017
Later among the works it cites.
A. Kuznetsova, H. Rom, N. Alldrin, J. Uijlings, I. Krasin, J. Pont-Tuset, S. Kamali, S. Popov, M. Malloci, T. Duerig et al. , “The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale,” arXiv preprint arXiv:1811.00982 , 2018
Original
2018
Later among the works it cites.
B. Wu, F. Jia, W. Liu, B. Ghanem, and S. Lyu, “Multi-label learning with missing labels using mixed dependency graphs,” International Journal of Computer Vision , vol. 126, no. 8, pp. 875–896, 2018
2018
Later among the works it cites.
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE transactions on pattern analysis and machine intelligence , vol. 40, no. 4, pp. 834–848, 2018
2018
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…