ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving

J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning . pmlr, 2015, pp. 2256–2265

2015

Earlier work this paper cites.

J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840–6851, 2020

2020

Earlier work this paper cites.

Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differential equations,” arXiv preprint arXiv:2011.13456 , 2020

Original

2020

Earlier work this paper cites.

H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuScenes: A Multimodal Dataset for Autonomous Driving,” in CVPR , 2020, pp. 11 621–11 631

2020

Earlier work this paper cites.

J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg, “Structured denoising diffusion models in discrete state-spaces,” Advances in neural information processing systems , vol. 34, pp. 17 981–17 993, 2021

2021

Earlier work this paper cites.

P. Wu, X. Jia, L. Chen, J. Yan, H. Li, and Y. Qiao, “Trajectory-guided control prediction for end-to-end autonomous driving: A simple yet strong baseline,” Advances in Neural Information Processing Systems , vol. 35, pp. 6119–6132, 2022

2022

Earlier work this paper cites.

C. Cui, Y. Ma, X. Cao, W. Ye, Y. Zhou, K. Liang, J. Chen, J. Lu, Z. Yang, K.-D. Liao, et al. , “A survey on multimodal large language models for autonomous driving,” arXiv preprint arXiv:2311.12320 , 2023

Original

2023

Earlier work this paper cites.

Z. Xu, Y. Zhang, E. Xie, Z. Zhao, Y. Guo, K. K. Y. Wong, Z. Li, and H. Zhao, “DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model,” Oct. 2023, arXiv:2310.01412

Original

2023

Earlier work this paper cites.

A. Lou, C. Meng, and S. Ermon, “Discrete diffusion modeling by estimating the ratios of the data distribution,” arXiv preprint arXiv:2310.16834 , 2023

Original

2023

Earlier work this paper cites.

C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao, “LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day,” June 2023, arXiv:2306.00890 [cs]. [Online]. Available: http://arxiv.org/abs/2306.00890

Original

2023

Earlier work this paper cites.

H. Zhang, X. Li, and L. Bing, “Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding,” in EMNLP . arXiv, 2023, arXiv:2306.02858 [cs, eess]. [Online]. Available: http://arxiv.org/abs/2306.02858

Original

2023

Earlier work this paper cites.

H. Zhang, X. Li, and L. Bing, “Video-llama: An instruction-tuned audio-visual language model for video understanding,” 2023. [Online]. Available: https://arxiv.org/abs/2306.02858

Original

2023

Earlier work this paper cites.

OpenAI, “GPT-4 Technical Report,” Mar. 2023, arXiv:2303.08774 [cs.CL]

Original

2023

Cited alongside, same era.

G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. , “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805 , 2023

Original

2023

Cited alongside, same era.

X. Tian, J. Gu, B. Li, Y. Liu, C. Hu, Y. Wang, K. Zhan, P. Jia, X. Lang, and H. Zhao, “DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models,” arXiv , 2024

2024

Cited alongside, same era.

C. Cui, Y. Ma, X. Cao, W. Ye, and Z. Wang, “Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles,” in 2024 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) . Waikoloa, HI, USA: IEEE, Jan. 2024, pp. 902–909. [Online]. Available: https://ieeexplore.ieee.org/document/10495655/

2024

S. Gong, S. Agarwal, Y. Zhang, J. Ye, L. Zheng, M. Li, C. An, P. Zhao, W. Bi, J. Han, et al. , “Scaling diffusion language models via adaptation from autoregressive models,” arXiv preprint arXiv:2410.17891 , 2024

Original

2024

Later among the works it cites.

C. Cui, Z. Yang, Y. Zhou, J. Peng, S.-Y. Park, C. Zhang, Y. Ma, X. Cao, W. Ye, Y. Feng, J. Panchal, L. Li, Y. Chen, and Z. Wang, “On-board vision-language models for personalized autonomous vehicle motion control: System design and real-world validation,” 2024. [Online]. Available: https://arxiv.org/abs/2411.11913

Original

2024

Later among the works it cites.

P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, and J. Lin, “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,” 2024. [Online]. Available: https://arxiv.org/abs/2409.12191

Original

2024

Later among the works it cites.

Z. You, S. Nie, X. Zhang, J. Hu, J. Zhou, Z. Lu, J.-R. Wen, and C. Li, “Llada-v: Large language diffusion models with visual instruction tuning,” 2025. [Online]. Available: https://arxiv.org/abs/2505.16933

Original