Well-designed prompts have demonstrated the potential to guide text-to-image models in generating amazing images. Although existing prompt engineering methods can provide high-level guidance, it is challenging for novice users to achieve the desired results by manually entering prompts due to a discrepancy between novice-user-input prompts and the model-preferred prompts. To bridge the distribution gap between user input behavior and model training datasets, we first construct a novel Coarse-Fine Granularity Prompts dataset (CFP) and propose a novel User-Friendly Fine-Grained Text Generation framework (UF-FGTG) for automated prompt optimization. For CFP, we construct a novel dataset for text-to-image tasks that combines coarse and fine-grained prompts to facilitate the development of automated prompt generation methods. For UF-FGTG, we propose a novel framework that automatically translates user-input prompts into model-preferred prompts. Specifically, we propose a prompt refiner that continually rewrites prompts to empower users to select results that align with their unique needs. Meanwhile, we integrate image-related loss functions from the text-to-image model into the training process of text generation to generate model-preferred prompts. Additionally, we propose an adaptive feature extraction module to ensure diversity in the generated results. Experiments demonstrate that our approach is capable of generating more visually appealing and diverse images than previous state-of-the-art methods, achieving an average improvement of 5% across six quality and aesthetic metrics.

翻译：精心设计的提示已展现出引导文图模型生成惊艳图像的潜力。现有提示工程方法虽能提供高层级指导，但由于新手用户输入提示与模型偏好提示之间存在差异，致使新手用户难以通过手动输入提示获得理想结果。为弥合用户输入行为与模型训练数据集之间的分布鸿沟，我们首先构建了全新的粗细粒度提示数据集(CFP)，并提出了一种面向自动化提示优化的用户友好型细粒度文本生成框架(UF-FGTG)。在CFP方面，我们针对文图任务构建了融合粗粒度与细粒度提示的新型数据集，以推动自动化提示生成方法的发展。在UF-FGTG方面，我们提出了自动将用户输入提示转化为模型偏好提示的全新框架。具体而言，我们设计了持续改写提示的提示精炼器，使用户能够选择契合个性化需求的结果；同时将文图模型中的图像相关损失函数集成到文本生成训练过程中，以生成模型偏好提示。此外，我们还提出了自适应特征提取模块，确保生成结果的多样性。实验证明，我们的方法能生成比现有最优方法更富视觉吸引力且更多样化的图像，在六项质量与美学指标上平均提升5%。