Current generative video models excel at producing novel content from text and image prompts, but leave a critical gap in editing existing pre-recorded videos, where minor alterations to the spoken script require preserving motion, temporal coherence, speaker identity, and accurate lip synchronization. We introduce EditYourself, a DiT-based framework for audio-driven video-to-video (V2V) editing that enables transcript-based modification of talking head videos, including the seamless addition, removal, and retiming of visually spoken content. Building on a general-purpose video diffusion model, EditYourself augments its V2V capabilities with audio conditioning and region-aware, edit-focused training extensions. This enables precise lip synchronization and temporally coherent restructuring of existing performances via spatiotemporal inpainting, including the synthesis of realistic human motion in newly added segments, while maintaining visual fidelity and identity consistency over long durations. This work represents a foundational step toward generative video models as practical tools for professional video post-production.

翻译：当前生成式视频模型在根据文本和图像提示生成新内容方面表现出色，但在编辑现有预录制视频方面存在关键不足——当需要对口述脚本进行细微修改时，必须同时保持动作连续性、时间一致性、说话人身份特征以及精确的唇形同步。本文提出EditYourself，一种基于扩散Transformer（DiT）的音频驱动视频到视频（V2V）编辑框架，支持基于转录文本的说话人视频修改，包括视觉语音内容的无缝添加、删除与时间重定位。该框架在通用视频扩散模型基础上，通过音频条件约束与区域感知的编辑导向训练扩展，增强了其V2V能力。这实现了精确的唇形同步与基于时空修复的现有表演时序重构，包括在新添加片段中合成逼真的人体动作，同时在长时程中保持视觉保真度与身份一致性。本工作标志着生成式视频模型向专业视频后期制作实用工具迈出了基础性一步。