SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Model

Understand

In the rapidly evolving area of image synthesis, a serious challenge is the presence of complex artifacts that compromise perceptual realism of synthetic images.

• To alleviate artifacts and improve quality of synthetic images, we fine-tune Vision-Language Model (VLM) as artifact classifier to automatically identify and classify a wide range of artifacts and provide supervision for further optimizing generative models.
• Specifically, we develop a comprehensive artifact taxonomy and construct a dataset of synthetic images with artifact annotations for fine-tuning VLM, named SynArtifact-1K.
• The fine-tuned VLM exhibits superior ability of identifying artifacts and outperforms the baseline by 25.66%.