Large-scale text-to-image generative models have made impressive strides, showcasing their ability to synthesize a vast array of high-quality images. However, adapting these models for artistic image editing presents two significant challenges. Firstly, users struggle to craft textual prompts that meticulously detail visual elements of the input image. Secondly, prevalent models, when effecting modifications in specific zones, frequently disrupt the overall artistic style, complicating the attainment of cohesive and aesthetically unified artworks. To surmount these obstacles, we build the innovative unified framework CreativeSynth, which is based on a diffusion model with the ability to coordinate multimodal inputs and multitask in the field of artistic image generation. By integrating multimodal features with customized attention mechanisms, CreativeSynth facilitates the importation of real-world semantic content into the domain of art through inversion and real-time style transfer. This allows for the precise manipulation of image style and content while maintaining the integrity of the original model parameters. Rigorous qualitative and quantitative evaluations underscore that CreativeSynth excels in enhancing artistic images' fidelity and preserves their innate aesthetic essence. By bridging the gap between generative models and artistic finesse, CreativeSynth becomes a custom digital palette.

翻译：大规模文本到图像生成模型取得了令人瞩目的进展，展现出合成海量高质量图像的能力。然而，将这些模型应用于艺术图像编辑面临两大关键挑战：首先，用户难以构建能够精确描述输入图像视觉元素的文本提示；其次，现有模型在修改特定区域时往往会破坏整体艺术风格，使得生成风格统一且具有美学价值的作品变得复杂。为克服这些障碍，我们构建了创新的统一框架CreativeSynth，该框架基于扩散模型，具备在艺术图像生成领域协调多模态输入与多任务处理的能力。通过将多模态特征与定制注意力机制相结合，CreativeSynth能够将现实语义内容通过反演与实时风格迁移导入艺术领域，从而在保持原始模型参数完整性的同时实现图像风格与内容的精准操控。严格的定性与定量评估表明，CreativeSynth在提升艺术图像保真度的同时，完美保留了其内在美学本质。通过弥合生成模型与艺术精妙性之间的鸿沟，CreativeSynth成为了一款定制化数字调色板。