Calliffusion: Chinese Calligraphy Generation and Style Transfer with Diffusion Modeling

Understand

In this paper, we propose Calliffusion, a system for generating high-quality Chinese calligraphy using diffusion models.

• Our model architecture is based on DDPM (Denoising Diffusion Probabilistic Models), and it is capable of generating common characters in five different scripts and mimicking the styles of famous calligraphers.
• Experiments demonstrate that our model can generate calligraphy that is difficult to distinguish from real artworks and that our controls for characters, scripts, and styles are effective.
• Moreover, we demonstrate one-shot transfer learning, using LoRA (Low-Rank Adaptation) to transfer Chinese calligraphy art styles to unseen characters and even out-of-domain symbols such as English letters and digits.