Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models

Understand

Text-to-image (T2I) research has grown explosively in the past year, owing to the large-scale pre-trained diffusion models and many emerging personalization and editing approaches.

• Yet, one pain point persists: the text prompt engineering, and searching high-quality text prompts for customized results is more art than science.
• Moreover, as commonly argued: "an image is worth a thousand words" - the attempt to describe a desired image with texts often ends up being ambiguous and cannot comprehensively cover delicate visual details, hence necessitating more additional controls from the visual domain.
• In this paper, we take a bold step forward: taking "Text" out of a pre-trained T2I diffusion model, to reduce the burdensome prompt engineering efforts for users.