AI-Generated Content (AIGC) has recently gained a surge in popularity, powered by its high efficiency and consistency in production, and its capability of being customized and diversified. The cross-modality nature of the representation learning mechanism in most AIGC technology allows for more freedom and flexibility in exploring new types of art that would be impossible in the past. Inspired by the pictogram subset of Chinese characters, we proposed PaCaNet, a CycleGAN-based pipeline for producing novel artworks that fuse two different art types, traditional Chinese painting and calligraphy. In an effort to produce stable and diversified output, we adopted three main technical innovations: 1. Using one-shot learning to increase the creativity of pre-trained models and diversify the content of the fused images. 2. Controlling the preference over generated Chinese calligraphy by freezing randomly sampled parameters in pre-trained models. 3. Using a regularization method to encourage the models to produce images similar to Chinese paintings. Furthermore, we conducted a systematic study to explore the performance of PaCaNet in diversifying fused Chinese painting and calligraphy, which showed satisfying results. In conclusion, we provide a new direction of creating arts by fusing the visual information in paintings and the stroke features in Chinese calligraphy. Our approach creates a unique aesthetic experience rooted in the origination of Chinese hieroglyph characters. It is also a unique opportunity to delve deeper into traditional artwork and, in doing so, to create a meaningful impact on preserving and revitalizing traditional heritage.
翻译:人工智能生成内容(AIGC)近年来因其高效稳定的生产能力、可定制化及多样化特性而迅速普及。大多数AIGC技术中表征学习机制的跨模态特性,使得探索以往难以实现的新型艺术形式拥有了更多自由度和灵活性。受汉字象形子集的启发,我们提出了PaCaNet——一种基于CycleGAN的流水线,用于生成融合中国传统绘画与书法两种艺术类型的新颖作品。为获得稳定且多样化的输出,我们采用了三项主要技术创新:1. 利用单样本学习增强预训练模型的创造力,并丰富融合图像的内容多样性;2. 通过冻结预训练模型中随机采样的参数,控制对生成书法的偏好;3. 采用正则化方法鼓励模型生成类似中国画的图像。此外,我们系统研究了PaCaNet在中国画与书法多样化融合中的性能,并取得了满意结果。总而言之,我们通过融合绘画的视觉信息与书法的笔画特征,为艺术创作提供了新方向。该方法植根于中国象形文字的起源,创造了独特的美学体验。同时,这也为深入探索传统艺术、并以有意义的方式保护与复兴传统遗产提供了独特契机。