Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles. While promising, adding realistic motions into these personalized images by text poses significant challenges in preserving distinct styles, high-fidelity details, and achieving motion controllability by text. In this paper, we present PIA, a Personalized Image Animator that excels in aligning with condition images, achieving motion controllability by text, and the compatibility with various personalized T2I models without specific tuning. To achieve these goals, PIA builds upon a base T2I model with well-trained temporal alignment layers, allowing for the seamless transformation of any personalized T2I model into an image animation model. A key component of PIA is the introduction of the condition module, which utilizes the condition frame and inter-frame affinity as input to transfer appearance information guided by the affinity hint for individual frame synthesis in the latent space. This design mitigates the challenges of appearance-related image alignment within and allows for a stronger focus on aligning with motion-related guidance.

翻译：近期个性化文本到图像（T2I）模型的进展彻底改变了内容创作方式，使非专业人士也能生成具有独特风格的惊艳图像。尽管前景广阔，但通过文本为这些个性化图像添加逼真运动，在保持独特风格、高保真细节以及实现文本可控运动方面仍面临重大挑战。本文提出PIA（个性化图像动画器），其在条件图像对齐、文本运动可控性以及无需特定调优即可兼容各类个性化T2I模型方面表现出色。为实现这些目标，PIA基于包含预训练时间对齐层的基础T2I模型构建，可将任意个性化T2I模型无缝转化为图像动画模型。PIA的关键创新在于引入条件模块，该模块以条件帧和帧间亲和力作为输入，在潜空间中通过亲和力提示引导外观信息传递，用于各帧的独立合成。该设计缓解了帧内外观相关图像对齐的挑战，并更专注于对齐运动相关引导信息。