Traffic Scene Generation from Natural Language Description for Autonomous Vehicles with Large Language Model

Understand

Generating realistic and controllable traffic scenes from natural language can greatly enhance the development and evaluation of autonomous driving systems.

• However, this task poses unique challenges: (1) grounding free-form text into spatially valid and semantically coherent layouts, (2) composing scenarios without predefined locations, and (3) planning multi-agent behaviors and selecting roads that respect agents' configurations.
• To address these, we propose a modular framework, TTSG, comprising prompt analysis, road retrieval, agent planning, and a novel plan-aware road ranking algorithm to solve these challenges.
• While large language models (LLMs) are used as general planners, our design integrates them into a tightly controlled pipeline that enforces structure, feasibility, and scene diversity.