CarLLaVA: Vision language models for camera-only closed-loop driving

Congested traffic states in empirical observations and microscopic simulations

Martin Treiber, Ansgar Hennecke, and Dirk Helbing · 2000

Earlier work this paper cites.

https://leaderboard.carla.org/ , 2020

Carla autonomous driving leaderboard · 2020

Earlier work this paper cites.

Masked autoencoders are scalable vision learners

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick · 2021

Earlier work this paper cites.

LoRA: Low-rank adaptation of large language models

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen · 2021

Earlier work this paper cites.

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever · 2021

Earlier work this paper cites.

An image is worth 16x16 words, what is a video worth?

Gilad Sharir, Asaf Noy, and Lihi Zelnik-Manor · 2021

Earlier work this paper cites.

Plant: Explainable planning transformers via object-level representations

Katrin Renz, Kashyap Chitta, Otniel-Bogdan Mercea, Almut Sophia Koepke, Zeynep Akata, and Andreas Geiger · 2022

Earlier work this paper cites.

Driving with llms: Fusing object-level vector modality for explainable autonomous driving, 2023

Long Chen, Oleg Sinavski, Jan Hünermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton · 2023

Earlier work this paper cites.

Drive like a human: Rethinking autonomous driving with large language models, 2023

Daocheng Fu, Xin Li, Licheng Wen, Min Dou, Pinlong Cai, Botian Shi, and Yu Qiao · 2023

Earlier work this paper cites.

Hidden biases of end-to-end driving models, 2023

Bernhard Jaeger, Kashyap Chitta, and Andreas Geiger · 2023

Cited alongside, same era.

Think twice before driving: Towards scalable decoders for end-to-end autonomous driving

Xiaosong Jia, Penghao Wu, Li Chen, Jiangwei Xie, Conghui He, Junchi Yan, and Hongyang Li · 2023

Cited alongside, same era.

BLIP-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi · 2023

Cited alongside, same era.

GPT-Driver: Learning to drive with gpt

Original

Jiageng Mao, Yuxi Qian, Hang Zhao, and Yue Wang · 2023

Cited alongside, same era.

LanguageMPC: Large language models as decision makers for autonomous driving

Original

Hao Sha, Yao Mu, Yuxuan Jiang, Li Chen, Chenfeng Xu, Ping Luo, Shengbo Eben Li, Masayoshi Tomizuka, Wei Zhan, and Mingyu Ding · 2023

https://wayve.ai/thinking/lingo-2-driving-with-language/ , 2024

Lingo-2: Driving with natural language · 2024

Closest in time.

https://medium.com/nuro/lambda-the-nuro-drivers-real-time-language-reasoning-model-7c3567b2d7b4 , 2024

Lambda: The nuro driver’s real time · 2024

Closest in time.

Pdm-lite: A rule-based planner for carla leaderboard 2.0

Jens Beißwenger · 2024

Closest in time.

Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks, 2024

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai · 2024

Closest in time.

Qlora: Efficient finetuning of quantized llms

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer · 2024

Closest in time.

Think2drive: Efficient reinforcement learning by thinking in latent world model for quasi-realistic autonomous driving (in carla-v2), 2024