2020

Rethinking the competition between detection and ReID in Multi-Object Tracking

Liang, Chao, Zhang, Zhipeng, Zhou, Xue et al.

Understand

Due to balanced accuracy and speed, one-shot models which jointly learn detection and identification embeddings, have drawn great attention in multi-object tracking (MOT).

However, the inherent differences and relations between detection and re-identification (ReID) are unconsciously overlooked because of treating them as two isolated tasks in the one-shot tracking paradigm.
This leads to inferior performance compared with existing two-stage methods.
In this paper, we first dissect the reasoning process for these two tasks, which reveals that the competition between them inevitably would destroy task-dependent representations learning.

Built on

A. Ess, B. Leibe, K. Schindler, and L. Van Gool, “A mobile vision system for robust multi-person tracking,” in 2008 IEEE Conference on Computer Vision and Pattern Recognition . IEEE, 2008, pp. 1–8
2008
Earlier work this paper cites.
K. Bernardin and R. Stiefelhagen, “Evaluating multiple object tracking performance: the clear mot metrics,” EURASIP Journal on Image and Video Processing , vol. 2008, pp. 1–10, 2008
2008
Earlier work this paper cites.
P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, “Object detection with discriminatively trained part-based models,” IEEE transactions on pattern analysis and machine intelligence , vol. 32, no. 9, pp. 1627–1645, 2009
2009
Earlier work this paper cites.
P. Dollár, C. Wojek, B. Schiele, and P. Perona, “Pedestrian detection: A benchmark,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition . IEEE, 2009, pp. 304–311
2009
Earlier work this paper cites.
Y. Li, C. Huang, and R. Nevatia, “Learning to associate: Hybridboosted multi-target tracker for crowded scene,” in 2009 IEEE conference on computer vision and pattern recognition . IEEE, 2009, pp. 2953–2960
2009
Earlier work this paper cites.
T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision . Springer, 2014, pp. 740–755
2014
Earlier work this paper cites.
F. Yu, W. Li, Q. Li, Y. Liu, X. Shi, and J. Yan, “Poi: Multiple object tracking with high performance detection and appearance feature,” in European Conference on Computer Vision . Springer, 2016, pp. 36–42
2016
Earlier work this paper cites.
A. Bewley, Z. Ge, L. Ott, F. Ramos, and B. Upcroft, “Simple online and realtime tracking,” in 2016 IEEE international conference on image processing (ICIP) . IEEE, 2016, pp. 3464–3468
2016
Earlier work this paper cites.
S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: towards real-time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence , vol. 39, no. 6, pp. 1137–1149, 2016
2016
Earlier work this paper cites.
F. Yang, W. Choi, and Y. Lin, “Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 2129–2137
2016
Earlier work this paper cites.
A. Milan, L. Leal-Taixé, I. Reid, S. Roth, and K. Schindler, “Mot16: A benchmark for multi-object tracking,” arXiv preprint arXiv:1603.00831 , 2016
Original
2016
Earlier work this paper cites.
E. Ristani, F. Solera, R. Zou, R. Cucchiara, and C. Tomasi, “Performance measures and a data set for multi-target, multi-camera tracking,” in European conference on computer vision . Springer, 2016, pp. 17–35
2016
Earlier work this paper cites.
N. Wojke, A. Bewley, and D. Paulus, “Simple online and realtime tracking with a deep association metric,” in 2017 IEEE international conference on image processing (ICIP) . IEEE, 2017, pp. 3645–3649
2017
Earlier work this paper cites.
T. Xiao, S. Li, B. Wang, L. Lin, and X. Wang, “Joint detection and identification feature learning for person search,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 3415–3424
2017
Earlier work this paper cites.
T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 2117–2125
2017
Earlier work this paper cites.
E. Bochinski, V. Eiselein, and T. Sikora, “High-speed tracking-by-detection without using image information,” in 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) . IEEE, 2017, pp. 1–6
2017
Earlier work this paper cites.

Similar

T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2980–2988
2017
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” arXiv preprint arXiv:1706.03762 , 2017
Original
2017
Cited alongside, same era.
S. Zhang, R. Benenson, and B. Schiele, “Citypersons: A diverse dataset for pedestrian detection,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 3213–3221
2017
Cited alongside, same era.
T. Xiao, S. Li, B. Wang, L. Lin, and X. Wang, “Joint detection and identification feature learning for person search,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 3415–3424
2017
Cited alongside, same era.
L. Zheng, H. Zhang, S. Sun, M. Chandraker, Y. Yang, and Q. Tian, “Person re-identification in the wild,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 1367–1376
2017
Cited alongside, same era.
Z. Zhou, J. Xing, M. Zhang, and W. Hu, “Online multi-target tracking with tensor-based high-order graph matching,” in 2018 24th International Conference on Pattern Recognition (ICPR) . IEEE, 2018, pp. 1809–1814
2018
Cited alongside, same era.
K. Fang, Y. Xiang, X. Li, and S. Savarese, “Recurrent autoregressive networks for online multi-object tracking,” in 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) . IEEE, 2018, pp. 466–475
2018
Cited alongside, same era.
J. Redmon and A. Farhadi, “Yolov3: An incremental improvement,” arXiv preprint arXiv:1804.02767 , 2018
Original
2018
Cited alongside, same era.
D. Xu, W. Ouyang, X. Wang, and N. Sebe, “Pad-net: Multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 675–684
2018
Cited alongside, same era.
S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “Cbam: Convolutional block attention module,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3–19
2018
Cited alongside, same era.
S. Shao, Z. Zhao, B. Li, T. Xiao, G. Yu, X. Zhang, and J. Sun, “Crowdhuman: A benchmark for detecting human in a crowd,” arXiv preprint arXiv:1805.00123 , 2018
Original
2018
Cited alongside, same era.
H. Luo, W. Jiang, Y. Gu, F. Liu, X. Liao, S. Lai, and J. Gu, “A strong baseline and batch normalization neck for deep person re-identification,” IEEE Transactions on Multimedia , vol. 22, no. 10, pp. 2597–2609, 2019
2019
Cited alongside, same era.

Then

P. Chu, H. Fan, C. C. Tan, and H. Ling, “Online multi-object tracking with instance-aware tracker and dynamic model refreshment,” in 2019 IEEE winter conference on applications of computer vision (WACV) . IEEE, 2019, pp. 161–170
2019
Later among the works it cites.
N. Mahmoudi, S. M. Ahadi, and M. Rahmati, “Multi-target tracking using cnn-based features: Cnnmtt,” Multimedia Tools and Applications , vol. 78, no. 6, pp. 7077–7096, 2019
2019
Later among the works it cites.
S. Zagoruyko and N. Komodakis, “Wide residual networks,” arXiv preprint arXiv:1605.07146 , 2016
Original
2019
Later among the works it cites.
W. Luo, J. Xing, A. Milan, X. Zhang, W. Liu, and T.-K. Kim, “Multiple object tracking: A literature review,” Artificial Intelligence , p. 103448, 2020
2020
Closest in time.
Z. Lu, V. Rathod, R. Votel, and J. Huang, “Retinatrack: Online single stage joint detection and tracking,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 14 668–14 678
2020
Closest in time.
Y. Zhang, C. Wang, X. Wang, W. Zeng, and W. Liu, “Fairmot: On the fairness of detection and re-identification in multiple object tracking,” arXiv e-prints , pp. arXiv–2004, 2020
2020
Closest in time.
G. Brasó and L. Leal-Taixé, “Learning a neural solver for multiple object tracking,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 6247–6257
2020
Closest in time.
J. Peng, C. Wang, F. Wan, Y. Wu, Y. Wang, Y. Tai, C. Wang, J. Li, F. Huang, and Y. Fu, “Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking,” in European Conference on Computer Vision . Springer, 2020, pp. 145–161
2020
Closest in time.
B. Pang, Y. Li, Y. Zhang, M. Li, and C. Lu, “Tubetk: Adopting tubes to track multi-object in a one-step training model,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 6308–6318
2020
Closest in time.
X. Zhou, V. Koltun, and P. Krähenbühl, “Tracking objects as points,” in European Conference on Computer Vision . Springer, 2020, pp. 474–490
2020
Closest in time.
Z. Zheng, P. Wang, W. Liu, J. Li, R. Ye, and D. Ren, “Distance-iou loss: Faster and better learning for bounding box regression,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 34, no. 07, 2020, pp. 12 993–13 000
2020
Closest in time.
P. Dendorfer, H. Rezatofighi, A. Milan, J. Shi, D. Cremers, I. Reid, S. Roth, K. Schindler, and L. Leal-Taixé, “Mot20: A benchmark for multi object tracking in crowded scenes,” arXiv preprint arXiv:2003.09003 , 2020
Original
2020
Closest in time.
T. Wang, K. Chen, W. Lin, J. See, Z. Zhang, Q. Xu, and X. Jia, “Spatio-temporal point process for multiple object tracking,” IEEE Transactions on Neural Networks and Learning Systems , 2020
2020
Closest in time.
J. Peng, T. Wang, W. Lin, J. Wang, J. See, S. Wen, and E. Ding, “Tpm: Multiple object tracking with tracklet-plane matching,” Pattern Recognition , vol. 107, p. 107480, 2020
2020
Closest in time.
B. Shuai, A. Berneshawi, M. Wang, C. Liu, D. Modolo, X. Li, and J. Tighe, “Application of multi-object tracking with siamese track-rcnn to the human in events dataset,” in Proceedings of the 28th ACM International Conference on Multimedia , 2020, pp. 4625–4629
2020
Closest in time.
W. Lin, H. Liu, S. Liu, Y. Li, R. Qian, T. Wang, N. Xu, H. Xiong, G.-J. Qi, and N. Sebe, “Human in events: A large-scale benchmark for human-centric video analysis in complex events,” arXiv preprint arXiv:2005.04490 , 2020
Original
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…