Learning Visual Generative Priors without Text

Understand

Although text-to-image (T2I) models have recently thrived as visual generative priors, their reliance on high-quality text-image pairs makes scaling up expensive.

• We argue that grasping the cross-modality alignment is not a necessity for a sound visual generative prior, whose focus should be on texture modeling.
• Such a philosophy inspires us to study image-to-image (I2I) generation, where models can learn from in-the-wild images in a self-supervised manner.
• We first develop a pure vision-based training framework, Lumos, and confirm the feasibility and the scalability of learning I2I models.