Recent advancements in personalized text-to-image (T2I) models have revolutionized content creation, empowering non-experts to generate stunning images with unique styles. While promising, adding realistic motions into these personalized images by text poses significant challenges in preserving distinct styles, high-fidelity details, and achieving motion controllability by text. In this paper, we present PIA, a Personalized Image Animator that excels in aligning with condition images, achieving motion controllability by text, and the compatibility with various personalized T2I models without specific tuning. To achieve these goals, PIA builds upon a base T2I model with well-trained temporal alignment layers, allowing for the seamless transformation of any personalized T2I model into an image animation model. A key component of PIA is the introduction of the condition module, which utilizes the condition frame and inter-frame affinity as input to transfer appearance information guided by the affinity hint for individual frame synthesis in the latent space. This design mitigates the challenges of appearance-related image alignment within and allows for a stronger focus on aligning with motion-related guidance.

翻译：近期个性化文本到图像（T2I）模型的进展彻底革新了内容创作领域，使非专业人士也能生成风格独特的惊艳图像。然而，尽管前景光明，通过文本为这些个性化图像添加真实运动仍面临显著挑战：既要保留原有风格、保持高保真细节，又要实现文本驱动的运动可控性。本文提出PIA——一种个性化图像动画生成器，它在条件图像对齐、文本驱动的运动可控性、以及无需特定调优即可兼容各类个性化T2I模型方面表现出色。为实现这些目标，PIA基于经过训练的时序对齐层的基础T2I模型构建，可将任意个性化T2I模型无缝转换为图像动画模型。其关键组件是条件模块，该模块以条件帧和帧间亲和度作为输入，在潜空间中利用亲和度线索引导外观信息传递，实现独立帧的合成。这种设计缓解了帧内外观相关图像对齐的挑战，并使其更专注于与运动相关引导的对齐任务。