In this paper, we propose Calliffusion, a system for generating high-quality Chinese calligraphy using diffusion models. Our model architecture is based on DDPM (Denoising Diffusion Probabilistic Models), and it is capable of generating common characters in five different scripts and mimicking the styles of famous calligraphers. Experiments demonstrate that our model can generate calligraphy that is difficult to distinguish from real artworks and that our controls for characters, scripts, and styles are effective. Moreover, we demonstrate one-shot transfer learning, using LoRA (Low-Rank Adaptation) to transfer Chinese calligraphy art styles to unseen characters and even out-of-domain symbols such as English letters and digits.
翻译:本文提出Calliffusion系统,该系统利用扩散模型生成高质量中国书法。模型架构基于DDPM(去噪扩散概率模型),能够生成五种不同书体的常用汉字,并模仿著名书法家的风格。实验表明,我们的模型能生成与真实艺术作品难以区分的书法作品,且在文字、书体和风格控制方面效果显著。此外,我们展示了单样本迁移学习,通过LoRA(低秩适配)将中国书法艺术风格迁移至未见过的文字,甚至包括英文字母和数字等域外符号。