Editing portrait videos is a challenging task that requires flexible yet precise control over a wide range of modifications, such as appearance changes, expression edits, or the addition of objects. The key difficulty lies in preserving the subject's original temporal behavior, demanding that every edited frame remains precisely synchronized with the corresponding source frame. We present Sync-LoRA, a method for editing portrait videos that achieves high-quality visual modifications while maintaining frame-accurate synchronization and identity consistency. Our approach uses an image-to-video diffusion model, where the edit is defined by modifying the first frame and then propagated to the entire sequence. To enable accurate synchronization, we train an in-context LoRA using paired videos that depict identical motion trajectories but differ in appearance. These pairs are automatically generated and curated through a synchronization-based filtering process that selects only the most temporally aligned examples for training. This training setup teaches the model to combine motion cues from the source video with the visual changes introduced in the edited first frame. Trained on a compact, highly curated set of synchronized human portraits, Sync-LoRA generalizes to unseen identities and diverse edits (e.g., modifying appearance, adding objects, or changing backgrounds), robustly handling variations in pose and expression. Our results demonstrate high visual fidelity and strong temporal coherence, achieving a robust balance between edit fidelity and precise motion preservation.
翻译:肖像视频编辑是一项具有挑战性的任务,需要在保持灵活性的同时精确控制多种修改,例如外观变化、表情调整或对象添加。主要难点在于保留主体的原始时序行为,要求每一帧编辑后的画面与对应的源帧保持精确同步。本文提出Sync-LoRA,一种用于肖像视频编辑的方法,能够在实现高质量视觉修改的同时,保持帧级精确同步与身份一致性。我们的方法采用图像到视频的扩散模型,通过修改首帧定义编辑内容,并将其传播至整个序列。为实现精确同步,我们使用成对视频训练一个上下文LoRA,这些视频描绘相同的运动轨迹但外观不同。这些成对数据通过基于同步性的过滤流程自动生成与筛选,仅选择时序对齐程度最高的样本用于训练。该训练机制使模型能够将源视频的运动线索与编辑首帧引入的视觉变化相结合。通过在紧凑且高度筛选的同步人像数据集上训练,Sync-LoRA能够泛化至未见过的身份和多样化编辑(如修改外观、添加对象或更换背景),并稳健处理姿态与表情的变化。实验结果表明,该方法具有高视觉保真度与强时序连贯性,在编辑保真度与精确运动保持之间实现了稳健平衡。