2021

Rethinking Global Context in Crowd Counting

Sun, Guolei, Liu, Yun, Probst, Thomas et al.

Understand

This paper investigates the role of global context for crowd counting.

Specifically, a pure transformer is used to extract features with global information from overlapping image patches.
Inspired by classification, we add a context token to the input sequence, to facilitate information exchange with tokens corresponding to image patches throughout transformer layers.
Due to the fact that transformers do not explicitly model the tried-and-true channel-wise interactions, we propose a token-attention module (TAM) to recalibrate encoded features through channel-wise attention informed by the context token.

Built on

T. Zhao and R. Nevatia, “Bayesian human segmentation in crowded situations,” in IEEE CVPR , vol. 2, 2003, pp. 452–459
2003
Earlier work this paper cites.
M. Li, Z. Zhang, K. Huang, and T. Tan, “Estimating the number of people in crowded scenes by mid based foreground segmentation and head-shoulder detection,” in IEEE ICPR , 2008, pp. 1–4
2008
Earlier work this paper cites.
C. Villani, Optimal transport: old and new . Springer Science & Business Media, 2008, vol. 338
2008
Earlier work this paper cites.
W. Ge and R. T. Collins, “Marked point processes for crowd counting,” in IEEE CVPR , 2009, pp. 2913–2920
2009
Earlier work this paper cites.
K. Chen, S. Gong, T. Xiang, and C. Change Loy, “Cumulative attribute space for age and crowd density estimation,” in IEEE CVPR , 2013, pp. 2467–2474
2013
Earlier work this paper cites.
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014
Original
2014
Earlier work this paper cites.
C. Zhang, H. Li, X. Wang, and X. Yang, “Cross-scene crowd counting via deep convolutional neural networks,” in IEEE CVPR , 2015, pp. 833–841
2015
Earlier work this paper cites.
K. Kopaczewski, M. Szczodrak, A. Czyzewski, and H. Krawczyk, “A method for counting people attending large public events,” Multimedia Tools and Applications , vol. 74, no. 12, pp. 4289–4301, 2015
2015
Earlier work this paper cites.
C. Wang, H. Zhang, L. Yang, S. Liu, and X. Cao, “Deep people counting in extremely dense crowds,” in ACM Multimedia , 2015, pp. 1299–1302
2015
Earlier work this paper cites.
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al. , “ImageNet large scale visual recognition challenge,” IJCV , vol. 115, no. 3, pp. 211–252, 2015
2015
Earlier work this paper cites.
Y. Zhang, D. Zhou, S. Chen, S. Gao, and Y. Ma, “Single-image crowd counting via multi-column convolutional neural network,” in IEEE CVPR , 2016, pp. 589–597
2016
Earlier work this paper cites.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE CVPR , 2016, pp. 770–778
2016
Earlier work this paper cites.
V. A. Sindagi and V. M. Patel, “CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting,” in IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) , 2017, pp. 1–6
2017
Earlier work this paper cites.
D. B. Sam, S. Surya, and R. V. Babu, “Switching convolutional neural network for crowd counting,” in IEEE CVPR . IEEE, 2017, pp. 4031–4039
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in NeurIPS , 2017, pp. 6000–6010
2017
Earlier work this paper cites.
A. Odena, C. Olah, and J. Shlens, “Conditional image synthesis with auxiliary classifier GANs,” in ICML . PMLR, 2017, pp. 2642–2651
2017
Earlier work this paper cites.
H. Idrees, M. Tayyab, K. Athrey, D. Zhang, S. Al-Maadeed, N. Rajpoot, and M. Shah, “Composition loss for counting, density map estimation and localization in dense crowds,” in ECCV , 2018, pp. 532–546
2018
Earlier work this paper cites.
D. B. Sam, N. N. Sajjan, R. V. Babu, and M. Srinivasan, “Divide and grow: Capturing huge diversity in crowd images with incrementally growing CNN,” in IEEE CVPR , 2018, pp. 3618–3626
2018
Earlier work this paper cites.
Y. Li, X. Zhang, and D. Chen, “CSRNet: Dilated convolutional neural networks for understanding the highly congested scenes,” in IEEE CVPR , 2018, pp. 1091–1100
2018
Earlier work this paper cites.
V. Ranjan, H. Le, and M. Hoai, “Iterative crowd counting,” in ECCV , 2018, pp. 270–285
2018
Earlier work this paper cites.
Z. Shi, L. Zhang, Y. Liu, X. Cao, Y. Ye, M.-M. Cheng, and G. Zheng, “Crowd counting with deep negative correlation learning,” in IEEE CVPR , 2018, pp. 5382–5390
2018
Earlier work this paper cites.
X. Cao, Z. Wang, Y. Zhao, and F. Su, “Scale aggregation network for accurate and efficient crowd counting,” in ECCV , 2018, pp. 734–750
2018
Earlier work this paper cites.

Similar

J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in IEEE CVPR , 2018, pp. 7132–7141
2018
Cited alongside, same era.
S. Woo, J. Park, J.-Y. Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” in ECCV , 2018, pp. 3–19
2018
Cited alongside, same era.
S. Gidaris, P. Singh, and N. Komodakis, “Unsupervised representation learning by predicting image rotations,” in ICLR , 2018
2018
Cited alongside, same era.
W. Liu, M. Salzmann, and P. Fua, “Context-aware crowd counting,” in IEEE CVPR , 2019, pp. 5099–5108
2019
Cited alongside, same era.
Q. Wang, J. Gao, W. Lin, and Y. Yuan, “Learning from synthetic data for crowd counting in the wild,” in IEEE CVPR , 2019, pp. 8198–8207
2019
Cited alongside, same era.
M. Shi, Z. Yang, C. Xu, and Q. Chen, “Revisiting perspective information for efficient crowd counting,” in IEEE CVPR , 2019, pp. 7279–7288
2019
Cited alongside, same era.
X. Jiang, Z. Xiao, B. Zhang, X. Zhen, X. Cao, D. Doermann, and L. Shao, “Crowd counting and density estimation by trellis encoder-decoder networks,” in IEEE CVPR , 2019, pp. 6133–6142
2019
Cited alongside, same era.
A. Zhang, L. Yue, J. Shen, F. Zhu, X. Zhen, X. Cao, and L. Shao, “Attentional neural fields for crowd counting,” in IEEE ICCV , 2019, pp. 5714–5723
2019
Cited alongside, same era.
J. Wan and A. Chan, “Adaptive density map generation for crowd counting,” in IEEE ICCV , 2019, pp. 1130–1139
2019
Cited alongside, same era.
Z. Shi, P. Mettes, and C. G. Snoek, “Counting with focus for free,” in IEEE ICCV , 2019, pp. 4200–4209
2019
Cited alongside, same era.
Z. Yan, Y. Yuan, W. Zuo, X. Tan, Y. Wang, S. Wen, and E. Ding, “Perspective-guided convolution networks for crowd counting,” in IEEE ICCV , 2019, pp. 952–961
2019
Cited alongside, same era.
H. Xiong, H. Lu, C. Liu, L. Liu, Z. Cao, and C. Shen, “From open set to closed set: Counting objects by spatial divide-and-conquer,” in IEEE ICCV , 2019, pp. 8362–8371
2019
Cited alongside, same era.

Then

J. Wan and A. Chan, “Modeling noisy annotations for crowd counting,” in NeurIPS , 2020, pp. 3386–3396
2020
Later among the works it cites.
B. Wang, H. Liu, D. Samaras, and M. Hoai, “Distribution matching for crowd counting,” in NeurIPS , 2020
2020
Later among the works it cites.
Z. Ma, X. Wei, X. Hong, and Y. Gong, “Learning scales from points: A scale-aware probabilistic model for crowd counting,” in ACM Multimedia , 2020, pp. 220–228
2020
Later among the works it cites.
J. He, X. Wu, J. Yang, and W. Hu, “Cpspnet: Crowd counting via semantic segmentation framework,” in 2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI) . IEEE, 2020, pp. 1104–1110
2020
Later among the works it cites.
V. Sindagi, R. Yasarla, and V. M. Patel, “JHU-CROWD++: Large-scale crowd counting dataset and a benchmark method,” IEEE TPAMI , 2020
2020
Later among the works it cites.
Q. Wang, J. Gao, W. Lin, and X. Li, “NWPU-Crowd: A large-scale benchmark for crowd counting and localization,” IEEE TPAMI , 2020
2020
Later among the works it cites.
J. Watson, O. Mac Aodha, D. Turmukhambetov, G. J. Brostow, and M. Firman, “Learning stereo from single images,” in ECCV , 2020, pp. 722–740
2020
Later among the works it cites.
G. Sun, W. Wang, J. Dai, and L. Van Gool, “Mining cross-image semantics for weakly supervised semantic segmentation,” in ECCV , 2020, pp. 347–365
2020
Later among the works it cites.
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in ECCV . Springer, 2020, pp. 213–229
2020
Later among the works it cites.
Z. Tian, C. Shen, and H. Chen, “Conditional convolutions for instance segmentation,” in ECCV , 2020, pp. 282–298
2020
Later among the works it cites.
Y. Yang, G. Li, Z. Wu, L. Su, Q. Huang, and N. Sebe, “Weakly-supervised crowd counting learns from sorting rather than locations,” in ECCV , 2020
2020
Later among the works it cites.
S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y. Wang, Y. Fu, J. Feng, T. Xiang, P. H. Torr, and L. Zhang, “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” in IEEE CVPR , 2021
2021
Closest in time.
G. Yang, H. Tang, M. Ding, N. Sebe, and E. Ricci, “Transformers solve the limited receptive field for monocular depth prediction,” arXiv preprint arXiv:2103.12091 , 2021
Original
2021
Closest in time.
E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” in NeurIPS , 2021
2021
Closest in time.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR , 2021
2021
Closest in time.
L. Yuan, Y. Chen, T. Wang, W. Yu, Y. Shi, F. E. Tay, J. Feng, and S. Yan, “Tokens-to-token ViT: Training vision transformers from scratch on imagenet,” arXiv preprint arXiv:2101.11986 , 2021
Original
2021
Closest in time.
C. Wang, Q. Song, B. Zhang, Y. Wang, Y. Tai, X. Hu, C. Wang, J. Li, J. Ma, and Y. Wu, “Uniformity in heterogeneity: Diving deep into count interval partition for crowd counting,” in IEEE ICCV , 2021, pp. 3234–3242
2021
Closest in time.
Q. Song, C. Wang, Z. Jiang, Y. Wang, Y. Tai, C. Wang, J. Li, F. Huang, and Y. Wu, “Rethinking counting and localization in crowds: A purely point-based framework,” in IEEE ICCV , 2021, pp. 3365–3374
2021
Closest in time.
X. Liu, G. Li, Z. Han, W. Zhang, Y. Yang, Q. Huang, and N. Sebe, “Exploiting sample correlation for crowd counting with multi-expert network,” in IEEE ICCV , 2021, pp. 3215–3224
2021
Closest in time.
D. Liang, X. Chen, W. Xu, Y. Zhou, and X. Bai, “TransCrowd: Weakly-supervised crowd counting with transformer,” arXiv preprint arXiv:2104.09116 , 2021
Original
2021
Closest in time.
Y. Lei, Y. Liu, P. Zhang, and L. Liu, “Towards using count-level weak supervision for crowd counting,” Pattern Recognition , vol. 109, p. 107616, 2021
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…