2021

BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View

Huang, Junjie, Huang, Guan, Zhu, Zheng et al.

Understand

Autonomous driving perceives its surroundings for decision making, which is one of the most complex scenarios in visual perception.

The success of paradigm innovation in solving the 2D object detection task inspires us to seek an elegant, feasible, and scalable paradigm for fundamentally pushing the performance boundary in this area.
To this end, we contribute the BEVDet paradigm in this paper.
BEVDet performs 3D object detection in Bird-Eye-View (BEV), where most target values are defined and route planning can be handily performed.

Built on

Rosenfeld, A., Thurston, M.: Edge and Curve Detection for Visual Scene Analysis. IEEE Transactions on computers 100
1971
Earlier work this paper cites.
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2012)
2012
Earlier work this paper cites.
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25
2012
Earlier work this paper cites.
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 580–587 (2014)
2014
Earlier work this paper cites.
Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common Objects in Context. In: Proceedings of the European Conference on Computer Vision. pp. 740–755. Springer (2014)
2014
Earlier work this paper cites.
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Advances in Neural Information Processing Systems 28
2015
Earlier work this paper cites.
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778 (2016)
2016
Earlier work this paper cites.
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You Only Look Once: Unified, Real-Time Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 779–788 (2016)
2016
Earlier work this paper cites.
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv preprint arXiv:1704.04861 (2017)
Original
2017
Earlier work this paper cites.
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely Connected Convolutional Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4700–4708 (2017)
2017
Earlier work this paper cites.
Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature Pyramid Networks for Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2117–2125 (2017)
2017
Earlier work this paper cites.
Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal Loss for Dense Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 2980–2988 (2017)
2017
Earlier work this paper cites.
Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. pp. 5105–5114 (2017)
2017
Earlier work this paper cites.
Liu, S., Qi, L., Qin, H., Shi, J., Jia, J.: Path Aggregation Network for Instance Segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8759–8768 (2018)
2018
Earlier work this paper cites.
Xiao, T., Liu, Y., Zhou, B., Jiang, Y., Sun, J.: Unified Perceptual Parsing for Scene Understanding. In: Proceedings of the European Conference on Computer Vision. pp. 418–434 (2018)
2018
Earlier work this paper cites.
Yan, Y., Mao, Y., Li, B.: SECOND: Sparsely Embedded Convolutional Detection. Sensors 18
2018
Earlier work this paper cites.
Zhou, Y., Tuzel, O.: VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4490–4499 (2018)
2018
Earlier work this paper cites.
Cai, Z., Vasconcelos, N.: Cascade R-CNN: High Quality Object Detection and Instance Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (2019)
2019
Earlier work this paper cites.
Chen, K., Pang, J., Wang, J., Xiong, Y., Li, X., Sun, S., Feng, W., Liu, Z., Shi, J., Ouyang, W., et al.: Hybrid Task Cascade for Instance Segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 4974–4983 (2019)
2019
Earlier work this paper cites.
Gao, S., Cheng, M.M., Zhao, K., Zhang, X.Y., Yang, M.H., Torr, P.H.: Res2Net: A New Multi-scale Backbone Architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence (2019)
2019
Earlier work this paper cites.
Ghiasi, G., Lin, T.Y., Le, Q.V.: NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7036–7045 (2019)
2019
Earlier work this paper cites.

Similar

Lang, A.H., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: PointPillars: Fast Encoders for Object Detection from Point Clouds. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 12697–12705 (2019)
2019
Cited alongside, same era.
Loshchilov, I., Hutter, F.: DECOUPLED WEIGHT DECAY REGULARIZATION. In: Proceedings of the International Conference on Learning Representations (2019)
2019
Cited alongside, same era.
Shao, S., Li, Z., Zhang, T., Peng, C., Yu, G., Zhang, X., Li, J., Sun, J.: Objects365: A Large-scale, High-quality Dataset for Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 8430–8439 (2019)
2019
Cited alongside, same era.
Simonelli, A., Bulo, S.R., Porzi, L., López-Antequera, M., Kontschieder, P.: Disentangling Monocular 3D Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 1991–1999 (2019)
2019
Cited alongside, same era.
Sun, K., Xiao, B., Liu, D., Wang, J.: Deep High-Resolution Representation Learning for Human Pose Estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 5693–5703 (2019)
2019
Cited alongside, same era.
Tan, M., Le, Q.: EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In: Proceedings of the International Conference on Machine Learning. pp. 6105–6114. PMLR (2019)
2019
Cited alongside, same era.
Tian, Z., Shen, C., Chen, H., He, T.: FCOS: Fully Convolutional One-Stage Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 9627–9636 (2019)
2019
Cited alongside, same era.
Zhou, X., Wang, D., Krähenbühl, P.: Objects as Points. arXiv preprint arXiv:1904.07850 (2019)
Original
2019
Cited alongside, same era.
Zhu, B., Jiang, Z., Zhou, X., Li, Z., Yu, G.: Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection. arXiv preprint arXiv:1908.09492 (2019)
Original
2019
Cited alongside, same era.
Caesar, H., Bankiti, V., Lang, A.H., Vora, S., Liong, V.E., Xu, Q., Krishnan, A., Pan, Y., Baldan, G., Beijbom, O.: nuScenes: A multimodal dataset for autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 11621–11631 (2020)
2020
Cited alongside, same era.
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-End Object Detection with Transformers. In: Proceedings of the European Conference on Computer Vision. pp. 213–229. Springer (2020)
2020
Cited alongside, same era.
Chen, Y., Liu, S., Shen, X., Jia, J.: DSGN: Deep Stereo Geometry Network for 3D Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 12536–12545 (2020)
2020
Cited alongside, same era.

Then

Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., Patnaik, V., Tsui, P., Guo, J., Zhou, Y., Chai, Y., Caine, B., et al.: Scalability in Perception for Autonomous Driving: Waymo Open Dataset. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 2446–2454 (2020)
2020
Later among the works it cites.
Yang, Z., Sun, Y., Liu, S., Jia, J.: 3DSSD: Point-based 3D Single Stage Object Detector. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 11040–11048 (2020)
2020
Later among the works it cites.
Zhang, S., Chi, C., Yao, Y., Lei, Z., Li, S.Z.: Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 9759–9768 (2020)
2020
Later among the works it cites.
Zhu, X., Ma, Y., Wang, T., Xu, Y., Shi, J., Lin, D.: SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds. In: European Conference on Computer Vision. pp. 581–597. Springer (2020)
2020
Later among the works it cites.
Kumar, A., Brazil, G., Liu, X.: GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8973–8983 (2021)
2021
Closest in time.
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In: Proceedings of the International Conference on Computer Vision. pp. 10012–10022 (2021)
2021
Closest in time.
Liu, Z., Zhou, D., Lu, F., Fang, J., Zhang, L.: AutoShape: Real-Time Shape-Aware Monocular 3D Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 15641–15650 (2021)
2021
Closest in time.
Lu, Y., Ma, X., Yang, L., Zhang, T., Liu, Y., Chu, Q., Yan, J., Ouyang, W.: Geometry Uncertainty Projection Network for Monocular 3D Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 3111–3121 (2021)
2021
Closest in time.
Nabati, R., Qi, H.: CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 1527–1536 (2021)
2021
Closest in time.
Park, D., Ambrus, R., Guizilini, V., Li, J., Gaidon, A.: Is Pseudo-Lidar needed for Monocular 3D Object detection? In: Proceedings of the International Conference on Computer Vision. pp. 3142–3152 (2021)
2021
Closest in time.
Reading, C., Harakeh, A., Chae, J., Waslander, S.L.: Categorical Depth Distribution Network for Monocular 3D Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8555–8564 (2021)
2021
Closest in time.
Wang, L., Du, L., Ye, X., Fu, Y., Guo, G., Xue, X., Feng, J., Zhang, L.: Depth-conditioned Dynamic Message Propagation for Monocular 3D Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 454–463 (2021)
2021
Closest in time.
Wang, L., Zhang, L., Zhu, Y., Zhang, Z., He, T., Li, M., Xue, X.: Progressive Coordinate Transforms for Monocular 3D Object Detection. In: Advances in Neural Information Processing Systems (2021)
2021
Closest in time.
Wang, T., Zhu, X., Pang, J., Lin, D.: FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detection. arXiv preprint arXiv:2104.10956 (2021)
Original
2021
Closest in time.
Wang, T., Zhu, X., Pang, J., Lin, D.: Probabilistic and Geometric Depth: Detecting Objects in Perspective. arXiv preprint arXiv:2107.14160 (2021)
Original
2021
Closest in time.
Wang, Y., Guizilini, V., Zhang, T., Wang, Y., Zhao, H., Solomon, J.: DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries. arXiv preprint arXiv:2110.06922 (2021)
Original
2021
Closest in time.
Yang, W., Li, Q., Liu, W., Yu, Y., Ma, Y., He, S., Pan, J.: Projecting Your View Attentively: Monocular Road Scene Layout Estimation via Cross-View Transformation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 15536–15545 (2021)
2021
Closest in time.
Yin, T., Zhou, X., Krahenbuhl, P.: Center-based 3D Object Detection and Tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 11784–11793 (2021)
2021
Closest in time.
Zhang, Y., Lu, J., Zhou, J.: Objects are Different: Flexible Monocular 3D Object Detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3289–3298 (2021)
2021
Closest in time.
Zhou, Y., He, Y., Zhu, H., Wang, C., Li, H., Jiang, Q.: Monocular 3D Object Detection: An Extrinsic Parameter Free Approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7556–7566 (2021)
2021
Closest in time.
Zou, Z., Ye, X., Du, L., Cheng, X., Tan, X., Zhang, L., Feng, J., Xue, X., Ding, E.: The Devil Is in the Task: Exploiting Reciprocal Appearance-Localization Features for Monocular 3D Object Detection. In: Proceedings of the International Conference on Computer Vision. pp. 2713–2722 (2021)
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…