Simulating the Visual World with Artificial Intelligence: A Roadmap

Understand

The landscape of video generation is shifting, from a focus on generating visually appealing clips to building virtual environments that support interaction and maintain physical plausibility.

• These developments point toward the emergence of video foundation models that function not only as visual generators but also as implicit world models, models that simulate the physical dynamics, agent-environment interactions, and task planning that govern real or imagined worlds.
• This survey provides a systematic overview of this evolution, conceptualizing modern video foundation models as the combination of two core components: an implicit world model and a video renderer.
• The world model encodes structured knowledge about the world, including physical laws, interaction dynamics, and agent behavior.