AutoDrive-QA: A Multiple-Choice Benchmark for Vision-Language Evaluation in Urban Autonomous Driving

Papineni, K., Roukos, S., Ward, T., and Zhu, W. J. (2002, July). BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL)

2002

Earlier work this paper cites.

Lin, C. Y. (2004, July). ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out

2004

Earlier work this paper cites.

Banerjee, S. and Lavie, A. (2005, June). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization

2005

Earlier work this paper cites.

Vedantam, R., Lawrence Zitnick, C., and Parikh, D. (2015). CIDEr: Consensus-based image description evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

2015

Earlier work this paper cites.

Anderson, P., Fernando, B., Johnson, M., and Gould, S. (2016). SPICE: Semantic propositional image caption evaluation. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part V

2016

Earlier work this paper cites.

Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, October). CARLA: An open urban driving simulator. In Proceedings of the Conference on Robot Learning (CoRL)

2017

Earlier work this paper cites.

Caesar, H., Bankiti, V., Lang, A. H., Vora, S., Liong, V. E., Xu, Q., and Beijbom, O. (2020). nuScenes: A multimodal dataset for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2020

Earlier work this paper cites.

He, P., Gao, J., and Chen, W. (2021). DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543

Original

2021

Earlier work this paper cites.

Lu, J., Ye, X., Ren, Y., and Yang, Y. (2022). Good, better, best: Textual distractors generation for multiple-choice visual question answering via reinforcement learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2022

Earlier work this paper cites.

Nie, M., Peng, R., Wang, C., Cai, X., Han, J., Xu, H., and Zhang, L. (2023). Reason2Drive: Towards interpretable and chain-based reasoning for autonomous driving. arXiv preprint arXiv:2312.03661

Original

2023

Earlier work this paper cites.

Wang, W., Xie, J., Hu, C., Zou, H., Fan, J., Tong, W., and Dai, J. (2023). DriveMLM: Aligning multi-modal large language models with behavioral planning states for autonomous driving. arXiv preprint arXiv:2312.09245

Original

2023

Earlier work this paper cites.

Mao, J., Ye, J., Qian, Y., Pavone, M., and Wang, Y. (2023). A language agent for autonomous driving. arXiv preprint arXiv:2311.10813

Original

2023

Earlier work this paper cites.

Wen, L., Yang, X., Fu, D., Wang, X., Cai, P., Li, X., Ma, T., Li, Y., Xu, L., Shang, D., Zhu, Z., Sun, S., Bai, Y., Cai, X., Dou, M., Hu, S., and Shi, B. (2023). On the road with GPT-4V(ision): Early explorations of visual–language model on autonomous driving. arXiv preprint arXiv:2310.01412

Original

2023

Cited alongside, same era.

Wang, W., Bao, H., Dong, L., Bjorck, J., Peng, Z., Liu, Q., Aggarwal, K., Mohammed, O. K., Singhal, S., Som, S., and Wei, F. (2023). Image as a foreign language: BEiT pretraining for vision and vision-language tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2023

Cited alongside, same era.

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M. A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. (2023). Mistral 7B. arXiv preprint arXiv:2310.06825

Original

2023

Cited alongside, same era.

OpenAI. (2023). GPT-4 technical report. arXiv preprint arXiv:2303.08774

Tian, X., Gu, J., Li, B., Liu, Y., Wang, Y., Zhao, Z., and Zhao, H. (2024). DriveVLM: The convergence of autonomous driving and large vision-language models. arXiv preprint arXiv:2402.12289

Original

2024

Later among the works it cites.

Fu, D., Lei, W., Wen, L., Cai, P., Mao, S., Dou, M., and Qiao, Y. (2024, June). Limsim++: A closed-loop platform for deploying multimodal LLMs in autonomous driving. In Proceedings of the 2024 IEEE Intelligent Vehicles Symposium (IV)

2024

Later among the works it cites.

Cao, X., Zhou, T., Ma, Y., Ye, W., Cui, C., Tang, K., and Zheng, C. (2024). MapLM: A real-world large-scale vision-language benchmark for map and traffic scene understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

2024

Later among the works it cites.

Qian, T., Chen, J., Zhuo, L., Jiao, Y., and Jiang, Y. G. (2024, March). NuScenes-QA: A multi-modal visual question answering benchmark for autonomous driving scenario. In Proceedings of the AAAI Conference on Artificial Intelligence