Text-to-image synthesis for the Chinese language poses unique challenges due to its large vocabulary size, and intricate character relationships. While existing diffusion models have shown promise in generating images from textual descriptions, they often neglect domain-specific contexts and lack robustness in handling the Chinese language. This paper introduces PAI-Diffusion, a comprehensive framework that addresses these limitations. PAI-Diffusion incorporates both general and domain-specific Chinese diffusion models, enabling the generation of contextually relevant images. It explores the potential of using LoRA and ControlNet for fine-grained image style transfer and image editing, empowering users with enhanced control over image generation. Moreover, PAI-Diffusion seamlessly integrates with Alibaba Cloud's Machine Learning Platform for AI, providing accessible and scalable solutions. All the Chinese diffusion model checkpoints, LoRAs, and ControlNets, including domain-specific ones, are publicly available. A user-friendly Chinese WebUI and the diffusers-api elastic inference toolkit, also open-sourced, further facilitate the easy deployment of PAI-Diffusion models in various environments, making it a valuable resource for Chinese text-to-image synthesis.

翻译：中文文本到图像合成因其庞大的词汇量和复杂的字符关系面临独特挑战。尽管现有扩散模型在从文本描述生成图像方面展现出潜力，但它们往往忽略领域特定上下文，且缺乏处理中文语言的鲁棒性。本文提出PAI-Diffusion——一个解决上述局限性的综合框架。PAI-Diffusion融合了通用与领域特定的中文扩散模型，可实现上下文相关图像的生成。该框架探索了利用LoRA和ControlNet实现细粒度图像风格迁移与编辑的潜力，赋予用户对图像生成的更强控制能力。此外，PAI-Diffusion无缝集成阿里云机器学习平台AI，提供可访问且可扩展的解决方案。所有中文扩散模型检查点、LoRA及ControlNet（含领域特定版本）均已公开。同时开源的友好型中文WebUI和diffusers-api弹性推理工具包，进一步促进了PAI-Diffusion模型在不同环境中的便捷部署，使其成为中文文生图合成的宝贵资源。