UrbanGenAI: Reconstructing Urban Landscapes using Panoptic Segmentation and Diffusion Models

Understand

In contemporary design practices, the integration of computer vision and generative artificial intelligence (genAI) represents a transformative shift towards more interactive and inclusive processes.

• These technologies offer new dimensions of image analysis and generation, which are particularly relevant in the context of urban landscape reconstruction.
• This paper presents a novel workflow encapsulated within a prototype application, designed to leverage the synergies between advanced image segmentation and diffusion models for a comprehensive approach to urban design.
• Our methodology encompasses the OneFormer model for detailed image segmentation and the Stable Diffusion XL (SDXL) diffusion model, implemented through ControlNet, for generating images from textual descriptions.