2018

Zero-Shot Object Detection

Bansal, Ankan, Sikka, Karan, Sharma, Gaurav et al.

Understand

We introduce and tackle the problem of zero-shot object detection (ZSD), which aims to detect object classes which are not observed during training.

We work with a challenging set of object classes, not restricting ourselves to similar and/or fine-grained categories as in prior works on zero-shot classification.
We present a principled approach by first adapting visual-semantic embeddings for ZSD.
We then discuss the problems associated with selecting a background class and motivate two background-aware approaches for learning robust detectors.

Built on

Miller, G.A.: Wordnet: a lexical database for english. Communications of the ACM
1995
Earlier work this paper cites.
Seeger, M.: Learning with labeled and unlabeled data. Tech. rep. (2000)
2000
Earlier work this paper cites.
Ferrari, V., Marin-Jimenez, M., Zisserman, A.: Pose search: retrieving people using their pose. In: CVPR. pp. 1–8. IEEE (2009)
2009
Earlier work this paper cites.
Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: CVPR. pp. 951–958. IEEE (2009)
2009
Earlier work this paper cites.
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. IJCV
2010
Earlier work this paper cites.
Lim, J.J., Salakhutdinov, R.R., Torralba, A.: Transfer learning by borrowing examples for multiclass object detection. In: NIPS. pp. 118–126 (2011)
2011
Earlier work this paper cites.
Liu, J., Kuipers, B., Savarese, S.: Recognizing human actions by attributes. In: CVPR. pp. 3337–3344. IEEE (2011)
2011
Earlier work this paper cites.
Qi, G.J., Aggarwal, C., Rui, Y., Tian, Q., Chang, S., Huang, T.: Towards cross-category knowledge propagation for learning visual concepts. In: CVPR. pp. 897–904. IEEE (2011)
2011
Earlier work this paper cites.
Fu, Y., Hospedales, T.M., Xiang, T., Gong, S.: Attribute learning for understanding unstructured social activity. In: ECCV. pp. 530–543. Springer (2012)
2012
Earlier work this paper cites.
Hoiem, D., Chodpathumwan, Y., Dai, Q.: Diagnosing error in object detectors. In: ECCV. pp. 340–353. Springer (2012)
2012
Earlier work this paper cites.
Parikh, D., Kovashka, A., Parkash, A., Grauman, K.: Relative attributes for enhanced human-machine communication. In: AAAI (2012)
2012
Earlier work this paper cites.
Akata, Z., Perronnin, F., Harchaoui, Z., Schmid, C.: Label-embedding for attribute-based classification. In: CVPR. pp. 819–826. IEEE (2013)
2013
Earlier work this paper cites.
Elhoseiny, M., Saleh, B., Elgammal, A.: Write a classifier: Zero-shot learning using purely textual descriptions. In: CVPR. pp. 2584–2591. IEEE (2013)
2013
Earlier work this paper cites.
Frome, A., Corrado, G.S., Shlens, J., Bengio, S., Dean, J., Mikolov, T.: Devise: A deep visual-semantic embedding model. In: NIPS. pp. 2121–2129 (2013)
2013
Earlier work this paper cites.
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Original
2013
Earlier work this paper cites.
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS. pp. 3111–3119 (2013)
2013
Earlier work this paper cites.
Norouzi, M., Mikolov, T., Bengio, S., Singer, Y., Shlens, J., Frome, A., Corrado, G.S., Dean, J.: Zero-shot learning by convex combination of semantic embeddings. arXiv preprint arXiv:1312.5650 (2013)
Original
2013
Earlier work this paper cites.
Socher, R., Ganjoo, M., Manning, C.D., Ng, A.: Zero-shot learning through cross-modal transfer. In: NIPS. pp. 935–943 (2013)
2013
Earlier work this paper cites.
Jain, L.P., Scheirer, W.J., Boult, T.E.: Multi-class open set recognition using probability of inclusion. In: ECCV. pp. 393–409. Springer (2014)
2014
Earlier work this paper cites.
Kingma, D., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Original
2014
Earlier work this paper cites.
Lampert, C.H., Nickisch, H., Harmeling, S.: Attribute-based classification for zero-shot visual object categorization. TPAMI
2014
Earlier work this paper cites.

Similar

Li, Z., Gavves, E., Mensink, T., Snoek, C.G.: Attributes make sense on segmented objects. In: ECCV. pp. 350–365. Springer (2014)
2014
Cited alongside, same era.
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: ECCV. pp. 740–755. Springer (2014)
2014
Cited alongside, same era.
Pennington, J., Socher, R., Manning, C.D.: Glove: Global vectors for word representation. In: EMNLP. vol. 14, pp. 1532–1543 (2014)
2014
Cited alongside, same era.
Zitnick, C.L., Dollár, P.: Edge boxes: Locating object proposals from edges. In: ECCV. pp. 391–405. Springer (2014)
2014
Cited alongside, same era.
Akata, Z., Reed, S., Walter, D., Lee, H., Schiele, B.: Evaluation of output embeddings for fine-grained image classification. In: CVPR. pp. 2927–2936. IEEE (2015)
2015
Cited alongside, same era.
Fu, Y., Yang, Y., Hospedales, T., Xiang, T., Gong, S.: Transductive multi-label zero-shot learning. arXiv preprint arXiv:1503.07790 (2015)
Original
2015
Cited alongside, same era.
Gavves, S., Mensink, T., Tommasi, T., Snoek, C., Tuytelaars, T.: Active transfer learning with zero-shot priors: Reusing past datasets for future tasks. In: ICCV. pp. 2731–2739. IEEE (2015)
2015
Cited alongside, same era.
Girshick, R.: Fast r-cnn. In: ICCV. pp. 1440–1448. IEEE (2015)
2015
Cited alongside, same era.
Ren, S., He, K., Girshick, R., Sun, J.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: NIPS. pp. 91–99 (2015)
2015
Cited alongside, same era.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recognition challenge. International Journal of Computer Vision
2015
Cited alongside, same era.
Anne Hendricks, L., Venugopalan, S., Rohrbach, M., Mooney, R., Saenko, K., Darrell, T., Mao, J., Huang, J., Toshev, A., Camburu, O., et al.: Deep compositional captioning: Describing novel object categories without paired training data. In: CVPR. pp. 1–10. IEEE (2016)
2016
Cited alongside, same era.
Bendale, A., Boult, T.E.: Towards open set deep networks. In: CVPR. pp. 1563–1572. IEEE (2016)
2016
Cited alongside, same era.

Then

Xu, B., Fu, Y., Jiang, Y.G., Li, B., Sigal, L.: Heterogeneous knowledge transfer in video emotion recognition, attribution and summarization. TPAMI (2016)
2016
Later among the works it cites.
Yu, R., Chen, X., Morariu, V.I., Davis, L.S.: The role of context selection in object detection. arXiv preprint arXiv:1609.02948 (2016)
Original
2016
Later among the works it cites.
Zhang, H., Shang, X., Yang, W., Xu, H., Luan, H., Chua, T.S.: Online collaborative learning for open-vocabulary visual classifiers. In: CVPR. pp. 2809–2817. IEEE (2016)
2016
Later among the works it cites.
Zhang, Z., Saligrama, V.: Zero-shot learning via joint latent similarity embedding. In: CVPR. pp. 6034–6042. IEEE (2016)
2016
Later among the works it cites.
Zhang, Z., Saligrama, V.: Zero-shot recognition via structured prediction. In: ECCV. pp. 533–548. Springer (2016)
2016
Later among the works it cites.
Aytar, Y., Vondrick, C., Torralba, A.: See, hear, and read: Deep aligned representations. arXiv preprint arXiv:1706.00932 (2017)
Original
2017
Later among the works it cites.
Faghri, F., Fleet, D.J., Kiros, J.R., Fidler, S.: Vse++: Improved visual-semantic embeddings. arXiv preprint arXiv:1707.05612 (2017)
Original
2017
Later among the works it cites.
Fu, Y., Xiang, T., Jiang, Y.G., Xue, X., Sigal, L., Gong, S.: Recent advances in zero-shot recognition. arXiv preprint arXiv:1710.04837 (2017)
Original
2017
Later among the works it cites.
Gupta, T., Shih, K., Singh, S., Hoiem, D.: Aligned image-word representations improve inductive transfer across vision-language tasks. arXiv preprint arXiv:1704.00260 (2017)
Original
2017
Later among the works it cites.
Kodirov, E., Xiang, T., Gong, S.: Semantic autoencoder for zero-shot learning. arXiv preprint arXiv:1704.08345 (2017)
Original
2017
Later among the works it cites.
Krasin, I., Duerig, T., Alldrin, N., Ferrari, V., Abu-El-Haija, S., Kuznetsova, A., Rom, H., Uijlings, J., Popov, S., Veit, A., Belongie, S., Gomes, V., Gupta, A., Sun, C., Chechik, G., Cai, D., Feng, Z., Narayanan, D., Murphy, K.: Openimages: A public dataset for large-scale multi-label and multi-class image classification. Dataset available from https://github.com/openimages (2017)
2017
Later among the works it cites.
Qiao, R., Liu, L., Shen, C., Hengel, A.v.d.: Visually aligned word embeddings for improving zero-shot learning. arXiv preprint arXiv:1707.05427 (2017)
Original
2017
Later among the works it cites.
Rahman, S., Khan, S.H., Porikli, F.: A unified approach for conventional zero-shot, generalized zero-shot and few-shot learning. arXiv preprint arXiv:1706.08653 (2017)
Original
2017
Later among the works it cites.
Sharma, G., Jurie, F., Schmid, C.: Expanded parts model for semantic description of humans in still images. TPAMI
2017
Later among the works it cites.
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI. pp. 4278–4284 (2017)
2017
Later among the works it cites.
Xian, Y., Lampert, C.H., Schiele, B., Akata, Z.: Zero-shot learning-a comprehensive evaluation of the good, the bad and the ugly. arXiv preprint arXiv:1707.00600 (2017)
Original
2017
Later among the works it cites.
Xu, H., Lv, X., Wang, X., Ren, Z., Bodla, N., Chellappa, R.: Deep regionlets for object detection. CoRR
Original
2017
Later among the works it cites.
Zhang, Y., Yuan, L., Guo, Y., He, Z., Huang, I.A., Lee, H.: Discriminative bimodal networks for visual localization and detection with natural language queries. arXiv preprint arXiv:1704.03944 (2017)
Original
2017
Later among the works it cites.
Rahman, S., Khan, S., Porikli, F.: Zero-shot object detection: Learning to simultaneously recognize and localize novel concepts. arXiv preprint arXiv:1803.06049 (2018)
Original
2018
Closest in time.
Zhu, P., Wang, H., Bolukbasi, T., Saligrama, V.: Zero-shot detection. arXiv preprint arXiv:1803.07113 (2018)
Original
2018
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…