The current state-of-the-art Diffusion model has demonstrated excellent results in generating images. However, the images are monotonous and are mostly the result of the distribution of images of people in the training set, making it challenging to generate multiple images for a fixed number of individuals. This problem can often only be solved by fine-tuning the training of the model. This means that each individual/animated character image must be trained if it is to be drawn, and the hardware and cost of this training is often beyond the reach of the average user, who accounts for the largest number of people. To solve this problem, the Character Image Feature Encoder model proposed in this paper enables the user to use the process by simply providing a picture of the character to make the image of the character in the generated image match the expectation. In addition, various details can be adjusted during the process using prompts. Unlike traditional Image-to-Image models, the Character Image Feature Encoder extracts only the relevant image features, rather than information about the model's composition or movements. In addition, the Character Image Feature Encoder can be adapted to different models after training. The proposed model can be conveniently incorporated into the Stable Diffusion generation process without modifying the model's ontology or used in combination with Stable Diffusion as a joint model.

翻译：当前最先进的扩散模型在图像生成方面展现出优异效果，但生成的图像较为单一，且主要源于训练集中人物图像分布的结果，难以针对固定数量的个体生成多张图像。该问题通常只能通过微调模型训练来解决，这意味着每绘制一个人物/动画角色图像都需要单独训练，而此类训练的硬件成本与开销往往超出占用户群体绝大多数的普通用户承受能力。为解决这一难题，本文提出的角色图像特征编码器模型使用户仅需提供角色图片即可使生成图像中的人物特征符合预期。此外，用户可在生成过程中通过提示词调整各类细节。与传统图像到图像模型不同，该编码器仅提取相关图像特征，而非构图或动作信息。同时，训练后的角色图像特征编码器可适配不同模型。本方案可便捷集成到Stable Diffusion生成流程中，无需修改模型本体，亦可作为联合模型与Stable Diffusion配合使用。