Multimodal Large Language Model Driven Scenario Testing for Autonomous Vehicles

D. Jung, D. Jung, C. Jeong, Y. Kou, and H. Peng, “Worst case scenarios generation and its application on driving,” SAE Technical Paper, Tech. Rep., 2007

2007

Earlier work this paper cites.

J. Zhou and L. del Re, “Reduced complexity safety testing for adas & adf,” IFAC-PapersOnLine , vol. 50, no. 1, pp. 5985–5990, 2017

2017

Earlier work this paper cites.

P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y.-P. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wiessner, “Microscopic Traffic Simulation using SUMO,” in 2018 21st International Conference on Intelligent Transportation Systems (ITSC) , pp. 2575–2582. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8569938

2018

Earlier work this paper cites.

G. E. Mullins, P. G. Stankiewicz, R. C. Hawthorne, and S. K. Gupta, “Adaptive generation of challenging scenarios for testing and evaluation of autonomous vehicles,” Journal of Systems and Software , vol. 137, pp. 197–215, 2018

2018

Earlier work this paper cites.

D. J. Fremont, T. Dreossi, S. Ghosh, X. Yue, A. L. Sangiovanni-Vincentelli, and S. A. Seshia, “Scenic: a language for scenario specification and scene generation,” in Proceedings of the 40th ACM SIGPLAN conference on programming language design and implementation , 2019, pp. 63–78

2019

Earlier work this paper cites.

P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” in Advances in Neural Information Processing Systems , vol. 33. Curran Associates, Inc., pp. 9459–9474. [Online]. Available: https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html

2020

Earlier work this paper cites.

D. Nalic, T. Mihalj, M. Bäumler, M. Lehmann, A. Eichberger, and S. Bernsteiner, “Scenario based testing of automated driving systems: A literature survey,” in FISITA web Congress , vol. 10, 2020, p. 1

2020

Earlier work this paper cites.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information processing systems , vol. 33, pp. 1877–1901, 2020

2020

Earlier work this paper cites.

J. Wang, A. Pun, J. Tu, S. Manivasagam, A. Sadat, S. Casas, M. Ren, and R. Urtasun, “Advsim: Generating safety-critical scenarios for self-driving vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 9909–9918

2021

Earlier work this paper cites.

M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman et al. , “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374 , 2021

Original

2021

Earlier work this paper cites.

Y. Xiao and W. Y. Wang, “On hallucination and predictive uncertainty in conditional language generation,” arXiv preprint arXiv:2103.15025 , 2021

Original

2021

Earlier work this paper cites.

Y. Li, D. Choi, J. Chung, N. Kushman, J. Schrittwieser, R. Leblond, T. Eccles, J. Keeling, F. Gimeno, A. Dal Lago et al. , “Competition-level code generation with alphacode,” Science , vol. 378, no. 6624, pp. 1092–1097, 2022

2022

Earlier work this paper cites.

D. Rempe, J. Philion, L. J. Guibas, S. Fidler, and O. Litany, “Generating useful accident-prone driving scenarios via a learned traffic prior,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . New Orleans, LA, USA: IEEE, Jun. 2022, p. 17284–17294. [Online]. Available: https://ieeexplore.ieee.org/document/9880074/

2022

Earlier work this paper cites.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

2022

Cited alongside, same era.

A. Neelakantan, T. Xu, R. Puri, A. Radford, J. M. Han, J. Tworek, Q. Yuan, N. Tezak, J. W. Kim, C. Hallacy et al. , “Text and code embeddings by contrastive pre-training,” arXiv preprint arXiv:2201.10005 , 2022

Original

2022

Cited alongside, same era.

S. Feng, H. Sun, X. Yan, H. Zhu, Z. Zou, S. Shen, and H. X. Liu, “Dense reinforcement learning for safety validation of autonomous vehicles,” Nature , vol. 615, no. 7953, pp. 620–627, 2023

2023

Cited alongside, same era.

S. Imani, L. Du, and H. Shrivastava, “Mathprompter: Mathematical reasoning using large language models,” arXiv preprint arXiv:2303.05398 , 2023

Original

2023

Cited alongside, same era.

K. Chitta, D. Dauner, and A. Geiger, “Sledge: Synthesizing simulation environments for driving agents with generative models,” arXiv preprint arXiv:2403.17933 , 2024

Original

2024

Closest in time.

C. Cui, Y. Ma, X. Cao, W. Ye, Y. Zhou, K. Liang, J. Chen, J. Lu, Z. Yang, K.-D. Liao et al. , “A survey on multimodal large language models for autonomous driving,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 958–979

2024

Closest in time.

D. Fu, X. Li, L. Wen, M. Dou, P. Cai, B. Shi, and Y. Qiao, “Drive like a human: Rethinking autonomous driving with large language models,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 910–919

2024

Closest in time.

J. Zhang, C. Xu, and B. Li, “Chatscene: Knowledge-enabled safety-critical scenario generation for autonomous vehicles,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 15 459–15 469