Large Language Models for Human-like Autonomous Driving: A Survey

D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of Go with deep neural networks and tree search,” Nature , vol. 529, no. 7587, pp. 484–489, 28 Jan. 2016. [Online]. Available: http://dx.doi.org/10.1038/nature16961

2016

Earlier work this paper cites.

N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transp. Res. Part A: Policy Pract. , vol. 94, pp. 182–193, 1 Dec. 2016

2016

Earlier work this paper cites.

J. Kim and J. Canny, “Interpretable learning for self-driving cars by visualizing causal attention,” in 2017 IEEE International Conference on Computer Vision (ICCV) . IEEE, Oct. 2017, pp. 2942–2950. [Online]. Available: http://openaccess.thecvf.com/content_iccv_2017/html/Kim_Interpretable_Learning_for_ICCV_2017_paper.html

2017

Earlier work this paper cites.

J. Guanetti, Y. Kim, and F. Borrelli, “Control of connected and automated vehicles: State of the art and future challenges,” Annu. Rev. Control , vol. 45, pp. 18–40, 1 Jan. 2018

2018

Earlier work this paper cites.

A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, “Improving language understanding by generative pre-training,” 11 Jun. 2018. [Online]. Available: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf

2018

Earlier work this paper cites.

J. Chen, B. Yuan, and M. Tomizuka, “Model-free Deep Reinforcement Learning for Urban Autonomous Driving,” in 2019 IEEE Intelligent Transportation Systems Conference (ITSC) . IEEE, Oct. 2019, pp. 2765–2771. [Online]. Available: http://dx.doi.org/10.1109/ITSC.2019.8917306

2019

Earlier work this paper cites.

W. Zeng, W. Luo, S. Suo, A. Sadat, B. Yang, S. Casas, and R. Urtasun, “End-to-end interpretable neural motion planner,” in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, Jun. 2019, pp. 8660–8669

2019

Earlier work this paper cites.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, and Others, “Language models are few-shot learners,” Adv. Neural Inf. Process. Syst. , vol. 33, pp. 1877–1901, 2020. [Online]. Available: https://papers.nips.cc/paper/2020/hash/1457c0d6bfcb4967418bfb8ac142f64a-Abstract.html

2020

Earlier work this paper cites.

H. J. Jeon, S. Milli, and A. Dragan, “Reward-rational (implicit) choice: A unifying formalism for reward learning,” in Conference on Neural Information Processing Systems , vol. abs/2002.04833, 1 Feb. 2020. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2020/file/2f10c1578a0706e06b6d7db6f0b4a6af-Paper.pdf

Original

2020

Earlier work this paper cites.

A. Aksjonov and V. Kyrki, “Rule-Based Decision-Making System for Autonomous Vehicles at Intersections with Mixed Traffic Environment,” in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) . IEEE, 19 Sep. 2021, pp. 660–666. [Online]. Available: http://dx.doi.org/10.1109/ITSC48978.2021.9565085

2021

Earlier work this paper cites.

E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-Rank Adaptation of Large Language Models,” in International Conference on Learning Representations , 6 Oct. 2021. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

2021

Earlier work this paper cites.

G. Qin and J. Eisner, “Learning How to Ask: Querying LMs with Mixtures of Soft Prompts,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, and Y. Zhou, Eds. Online: Association for Computational Linguistics, Jun. 2021, pp. 5203–5212. [Online]. Available: https://aclanthology.org/2021.naacl-main.410

2021

Earlier work this paper cites.

S. Aradi, “Survey of deep reinforcement learning for motion planning of autonomous vehicles,” IEEE Trans. Intell. Transp. Syst. , vol. 23, no. 2, pp. 740–759, Feb. 2022

2022

Earlier work this paper cites.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou, “Chain-of-thought prompting elicits reasoning in large language models,” in Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 27 Jan. 2022, pp. 24 824–24 837

2022

Earlier work this paper cites.

J. Wei, M. Bosma, V. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le, “Finetuned Language Models are Zero-Shot Learners,” in International Conference on Learning Representations , 6 Oct. 2022. [Online]. Available: https://openreview.net/forum?id=gEZrGCozdqR

2022

Earlier work this paper cites.

J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, and E. Rutherford, “Flamingo: a Visual Language Model for Few-Shot Learning,” in 36th Conference on Neural Information Processing Systems . [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/960a172bc7fbf0177ccccbb411a7d800-Paper-Conference.pdf

2022

Earlier work this paper cites.

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe, “Training language models to follow instructions with human feedback,” in Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 4 Mar. 2022, pp. 27 730–27 744. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2022/file/b1efde53be364a73914f58805a001731-Paper-Conference.pdf

2022

Earlier work this paper cites.

S. M. Xie, A. Raghunathan, P. Liang, and T. Ma, “An Explanation of In-context Learning as Implicit Bayesian Inference,” in International Conference on Learning Representations . openreview.net, 2022. [Online]. Available: https://openreview.net/pdf?id=RdJVFCHjUMI

2022

Earlier work this paper cites.

“Microsoft Responsible AI Standard, v2,” Tech. Rep., Jun. 2022. [Online]. Available: https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE5cmFl?culture=en-us&country=us

2022

Earlier work this paper cites.

K. Yuan, Y. Huang, S. Yang, M. Wu, D. Cao, Q. Chen, and H. Chen, “Evolutionary Decision-Making and Planning for Autonomous Driving: A Hybrid Augmented Intelligence Framework,” IEEE Trans. Intell. Transp. Syst. , vol. PP, no. 99, pp. 1–13, 2023. [Online]. Available: http://dx.doi.org/10.1109/TITS.2023.3349198

2023

Cited alongside, same era.

R. Zhao, Y. Li, F. Gao, Z. Gao, and T. Zhang, “Multi-Agent Constrained Policy Optimization for Conflict-Free Management of Connected Autonomous Vehicles at Unsignalized Intersections,” IEEE Trans. Intell. Transp. Syst. , vol. PP, no. 99, pp. 1–15. [Online]. Available: http://dx.doi.org/10.1109/TITS.2023.3331723

2023

Cited alongside, same era.

S. Feng, H. Sun, X. Yan, H. Zhu, Z. Zou, S. Shen, and H. X. Liu, “Dense reinforcement learning for safety validation of autonomous vehicles,” Nature , vol. 615, no. 7953, pp. 620–627, Mar. 2023. [Online]. Available: http://dx.doi.org/10.1038/s41586-023-05732-2

2023

Cited alongside, same era.

K. Tanahashi, Y. Inoue, Y. Yamaguchi, H. Yaginuma, D. Shiotsuka, H. Shimatani, K. Iwamasa, Y. Inoue, T. Yamaguchi, K. Igari, T. Horinouchi, K. Tokuhiro, Y. Tokuchi, and S. Aoki, “Evaluation of Large Language Models for Decision Making in Autonomous Driving,” in 2023 Symposium on Machine Learning for Autonomous Driving collocated with NeurIPS , 11 Dec. 2023. [Online]. Available: http://arxiv.org/abs/2312.06351

Original

2023

Later among the works it cites.

W. Wang, J. Xie, C. Hu, H. Zou, J. Fan, W. Tong, Y. Wen, S. Wu, H. Deng, Z. Li, H. Tian, L. Lu, X. Zhu, X. Wang, Y. Qiao, and J. Dai, “DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving,” 14 Dec. 2023. [Online]. Available: http://arxiv.org/abs/2312.09245

Original

2023

Later among the works it cites.

D. Fu, X. Li, L. Wen, M. Dou, P. Cai, B. Shi, and Y. Qiao, “Drive Like a Human: Rethinking Autonomous Driving with Large Language Models,” 14 Jul. 2023. [Online]. Available: http://arxiv.org/abs/2307.07162

Original

2023

Later among the works it cites.

Y. Jin, X. Shen, H. Peng, X. Liu, J. Qin, J. Li, J. Xie, P. Gao, G. Zhou, and J. Gong, “SurrealDriver: Designing Generative Driver Agent Simulation Framework in Urban Contexts based on Large Language Model,” 22 Sep. 2023. [Online]. Available: http://arxiv.org/abs/2309.13193