Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles. While promising, adding realistic motions into these personalized images by text poses significant challenges in preserving distinct styles, high-fidelity details, and achieving motion controllability by text. In this paper, we present PIA, a Personalized Image Animator that excels in aligning with condition images, achieving motion controllability by text, and the compatibility with various personalized T2I models without specific tuning. To achieve these goals, PIA builds upon a base T2I model with well-trained temporal alignment layers, allowing for the seamless transformation of any personalized T2I model into an image animation model. A key component of PIA is the introduction of the condition module, which utilizes the condition frame and inter-frame affinity as input to transfer appearance information guided by the affinity hint for individual frame synthesis in the latent space. This design mitigates the challenges of appearance-related image alignment within and allows for a stronger focus on aligning with motion-related guidance.

翻译：近年来，个性化文本到图像（T2I）模型的进步彻底改变了内容创作，使非专业人士也能生成具有独特风格的惊艳图像。然而，通过文本为这些个性化图像添加逼真运动仍面临显著挑战，包括保持独特风格、高保真细节以及通过文本实现运动可控性。本文提出PIA（个性化图像动画器），它在与条件图像对齐、通过文本实现运动可控性以及兼容多种个性化T2I模型（无需特定调优）方面表现出色。为实现这些目标，PIA基于一个带有训练良好时间对齐层的基座T2I模型构建，可将任意个性化T2I模型无缝转化为图像动画模型。PIA的关键组成部分是引入条件模块，该模块以条件帧和帧间亲和度为输入，在潜在空间中通过亲和度提示传递外观信息以指导单帧合成。这一设计减轻了帧间外观对齐的挑战，并更专注于与运动相关引导的对齐。