2021

TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking

Chu, Peng, Wang, Jiang, You, Quanzeng et al.

Understand

Tracking multiple objects in videos relies on modeling the spatial-temporal interactions of the objects.

In this paper, we propose a solution named TransMOT, which leverages powerful graph transformers to efficiently model the spatial and temporal interactions among the objects.
TransMOT effectively models the interactions of a large number of objects by arranging the trajectories of the tracked objects as a set of sparse weighted graphs, and constructing a spatial graph transformer encoder layer, a temporal transformer encoder layer, and a spatial graph transformer decoder layer based on the graphs.
TransMOT is not only more computationally efficient than the traditional Transformer, but it also achieves better tracking accuracy.

Built on

Evaluating multiple object tracking performance: the clear mot metrics
Keni Bernardin and Rainer Stiefelhagen · 2008
Earlier work this paper cites.
Robust object tracking by hierarchical association of detection responses
Chang Huang, Bo Wu, and Ramakant Nevatia · 2008
Earlier work this paper cites.
Global data association for multi-object tracking using network flows
Li Zhang, Yuan Li, and Ramakant Nevatia · 2008
Earlier work this paper cites.
Object detection with discriminatively trained part-based models
Pedro F Felzenszwalb, Ross B Girshick, David McAllester, and Deva Ramanan · 2010
Earlier work this paper cites.
Globally-optimal greedy algorithms for tracking a variable number of objects
Hamed Pirsiavash, Deva Ramanan, and Charless C Fowlkes · 2011
Earlier work this paper cites.
Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs
Amir Roshan Zamir, Afshin Dehghan, and Mubarak Shah · 2012
Earlier work this paper cites.
Multi-target tracking by rank-1 tensor approximation
Xinchu Shi, Haibin Ling, Junliang Xing, and Weiming Hu · 2013
Earlier work this paper cites.
Target identity-aware network flow for online multiple target tracking
Afshin Dehghan, Yicong Tian, Philip HS Torr, and Mubarak Shah · 2015
Earlier work this paper cites.
Multiple hypothesis tracking revisited
Chanho Kim, Fuxin Li, Arridhana Ciptadi, and James M Rehg · 2015
Earlier work this paper cites.
MOTChallenge 2015: Towards a benchmark for multi-target tracking
Original
Laura Leal-Taixé, Anton Milan, Ian Reid, Stefan Roth, and Konrad Schindler · 2015
Earlier work this paper cites.
Faster r-cnn: towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun · 2015
Earlier work this paper cites.
Learning to track: Online multi-object tracking by decision making
Yu Xiang, Alexandre Alahi, and Silvio Savarese · 2015
Earlier work this paper cites.
Simple online and realtime tracking
Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, and Ben Upcroft · 2016
Earlier work this paper cites.
Convolutional neural networks on graphs with fast localized spectral filtering
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst · 2016
Earlier work this paper cites.
Improving multi-frame data association with sparse representations for robust near-online multi-object tracking
Loïc Fagot-Bouquet, Romaric Audigier, Yoann Dhome, and Frédéric Lerasle · 2016
Earlier work this paper cites.
Cdt: Cooperative detection and tracking for tracing multiple objects in video sequences
Han-Ul Kim and Chang-Su Kim · 2016
Earlier work this paper cites.
MOT16: A benchmark for multi-object tracking
Original
A. Milan, L. Leal-Taixé, I. Reid, S. Roth, and K. Schindler · 2016
Earlier work this paper cites.
Multi-target tracking by discrete-continuous energy minimization
Anton Milan, Konrad Schindler, and Stefan Roth · 2016
Earlier work this paper cites.
Deep tracking: Seeing beyond seeing using recurrent neural networks
Peter Ondruska and Ingmar Posner · 2016
Earlier work this paper cites.
Performance measures and a data set for multi-target, multi-camera tracking
Ergys Ristani, Francesco Solera, Roger Zou, Rita Cucchiara, and Carlo Tomasi · 2016
Earlier work this paper cites.
Multi-person tracking by multicut and deep matching
Siyu Tang, Bjoern Andres, Mykhaylo Andriluka, and Bernt Schiele · 2016
Earlier work this paper cites.

Similar

Tracking interacting objects using intertwined flows
Xinchao Wang, Engin Türetken, Francois Fleuret, and Pascal Fua · 2016
Cited alongside, same era.
Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers
Fan Yang, Wongun Choi, and Yuanqing Lin · 2016
Cited alongside, same era.
High-speed tracking-by-detection without using image information
Erik Bochinski, Volker Eiselein, and Thomas Sikora · 2017
Cited alongside, same era.
Enhancing detection model for multiple hypothesis tracking
Jiahui Chen, Hao Sheng, Yang Zhang, and Zhang Xiong · 2017
Cited alongside, same era.
Pathtrack: Fast trajectory annotation with path supervision
Santiago Manen, Michael Gygli, Dengxin Dai, and Luc Van Gool · 2017
Cited alongside, same era.
Online multi-target tracking using recurrent neural networks
Anton Milan, Seyed Hamid Rezatofighi, Anthony R Dick, Ian D Reid, and Konrad Schindler · 2017
Cited alongside, same era.
Tracking the untrackable: Learning to track multiple cues with long-term dependencies
Original
Amir Sadeghian, Alexandre Alahi, and Silvio Savarese · 2017
Cited alongside, same era.
Multiple people tracking by lifted multicut and person re-identification
Siyu Tang, Mykhaylo Andriluka, Bjoern Andres, and Bernt Schiele · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Simple online and realtime tracking with a deep association metric
Nicolai Wojke, Alex Bewley, and Dietrich Paulus · 2017
Cited alongside, same era.
Confidence-based data association and discriminative deep appearance learning for robust online multi-object tracking
Seung-Hwan Bae and Kuk-Jin Yoon · 2018
Cited alongside, same era.
Recurrent autoregressive networks for online multi-object tracking
Kuan Fang, Yu Xiang, Xiaocheng Li, and Silvio Savarese · 2018
Cited alongside, same era.

Then

Learning a neural solver for multiple object tracking
Guillem Brasó and Laura Leal-Taixé · 2020
Later among the works it cites.
End-to-end object detection with transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko · 2020
Later among the works it cites.
Mat: Motion-aware multi-object tracking
Original
Shoudong Han, Piao Huang, Hongwei Wang, En Yu, Donghaisheng Liu, Xiaofeng Pan, and Jun Zhao · 2020
Later among the works it cites.
Lifted disjoint paths with application in multiple object tracking
Andrea Hornakova, Roberto Henschel, Bodo Rosenhahn, and Paul Swoboda · 2020
Later among the works it cites.
Rethinking the competition between detection and reid in multi-object tracking
Original
Chao Liang, Zhipeng Zhang, Yi Lu, Xue Zhou, Bing Li, Xiyong Ye, and Jianxiao Zou · 2020
Later among the works it cites.
Polytransform: Deep polygon transformer for instance segmentation
Justin Liang, Namdar Homayounfar, Wei-Chiu Ma, Yuwen Xiong, Rui Hu, and Raquel Urtasun · 2020
Later among the works it cites.
Human in events: A large-scale benchmark for human-centric video analysis in complex events
Original
Weiyao Lin, Huabin Liu, Shizhan Liu, Yuxi Li, Guo-Jun Qi, Rui Qian, Tao Wang, Nicu Sebe, Ning Xu, Hongkai Xiong, et al · 2020
Later among the works it cites.
Multiple object tracking: A literature review
Wenhan Luo, Junliang Xing, Anton Milan, Xiaoqin Zhang, Wei Liu, and Tae-Kyun Kim · 2020
Later among the works it cites.
Tubetk: Adopting tubes to track multi-object in a one-step training model
Bo Pang, Yizhuo Li, Yifan Zhang, Muchen Li, and Cewu Lu · 2020
Later among the works it cites.
Chained-tracker: Chaining paired attentive regression results for end-to-end joint multiple-object detection and tracking
Jinlong Peng, Changan Wang, Fangbin Wan, Yang Wu, Yabiao Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Yanwei Fu · 2020
Later among the works it cites.
Fgagt: Flow-guided adaptive graph tracking
Original
Chaobing Shan, Chunbo Wei, Bing Deng, Jianqiang Huang, Xian-Sheng Hua, Xiaoliang Cheng, and Kewei Liang · 2020
Later among the works it cites.
Transtrack: Multiple-object tracking with transformer
Original
Peize Sun, Yi Jiang, Rufeng Zhang, Enze Xie, Jinkun Cao, Xinting Hu, Tao Kong, Zehuan Yuan, Changhu Wang, and Ping Luo · 2020
Later among the works it cites.
Joint detection and multi-object tracking with graph neural networks
Original
Yongxin Wang, Xinshuo Weng, and Kris Kitani · 2020
Later among the works it cites.
How to train your deep multi-object tracker
Yihong Xu, Aljosa Osep, Yutong Ban, Radu Horaud, Laura Leal-Taixé, and Xavier Alameda-Pineda · 2020
Later among the works it cites.
Spatio-temporal graph transformer networks for pedestrian trajectory prediction
Cunjun Yu, Xiao Ma, Jiawei Ren, Haiyu Zhao, and Shuai Yi · 2020
Later among the works it cites.
Multiplex labeling graph for near-online tracking in crowded scenes
Yang Zhang, Hao Sheng, Yubin Wu, Shuai Wang, Wei Ke, and Zhang Xiong · 2020
Later among the works it cites.
Fairmot: On the fairness of detection and re-identification in multiple object tracking
Yifu Zhang, Chunyu Wang, Xinggang Wang, Wenjun Zeng, and Wenyu Liu · 2020
Later among the works it cites.
Tracking objects as points
Xingyi Zhou, Vladlen Koltun, and Philipp Krähenbühl · 2020
Later among the works it cites.
Learning a proposal classifier for multiple object tracking
Peng Dai, Renliang Weng, Wongun Choi, Changshui Zhang, Zhangping He, and Wei Ding · 2021
Closest in time.
Trackformer: Multi-object tracking with transformers
Original
Tim Meinhardt, Alexander Kirillov, Laura Leal-Taixe, and Christoph Feichtenhofer · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…