Despite the fact that text-to-video (TTV) model has recently achieved remarkable success, there have been few approaches on TTV for its extension to video editing. Motivated by approaches on TTV models adapting from diffusion-based text-to-image (TTI) models, we suggest the video editing framework given only a pretrained TTI model and a single <text, video> pair, which we term Edit-A-Video. The framework consists of two stages: (1) inflating the 2D model into the 3D model by appending temporal modules and tuning on the source video (2) inverting the source video into the noise and editing with target text prompt and attention map injection. Each stage enables the temporal modeling and preservation of semantic attributes of the source video. One of the key challenges for video editing include a background inconsistency problem, where the regions not included for the edit suffer from undesirable and inconsistent temporal alterations. To mitigate this issue, we also introduce a novel mask blending method, termed as sparse-causal blending (SC Blending). We improve previous mask blending methods to reflect the temporal consistency so that the area where the editing is applied exhibits smooth transition while also achieving spatio-temporal consistency of the unedited regions. We present extensive experimental results over various types of text and videos, and demonstrate the superiority of the proposed method compared to baselines in terms of background consistency, text alignment, and video editing quality.

翻译：尽管文本到视频（TTV）模型近期取得了显著成功，但将其扩展至视频编辑任务的方法仍相对匮乏。受基于扩散的文本到图像（TTI）模型衍生的TTV方法启发，我们提出一个仅需预训练TTI模型和单个<文本，视频>对的视频编辑框架，命名为Edit-A-Video。该框架包含两个阶段：（1）通过附加时序模块将2D模型扩展为3D模型，并在源视频上进行微调；（2）将源视频反转为噪声，并通过目标文本提示与注意力图注入进行编辑。每个阶段均实现了源视频的时序建模与语义属性保持。视频编辑的关键挑战之一是背景不一致问题——即未参与编辑的区域会出现不期望的、不一致的时序变化。为缓解此问题，我们进一步提出一种新颖的掩膜混合方法——稀疏因果混合（SC Blending）。该方法改进了现有掩膜混合技术以反映时序一致性，使编辑区域实现平滑过渡，同时保持未编辑区域的时空一致性。我们针对多种文本与视频类型开展了大量实验，结果表明，所提方法在背景一致性、文本对齐度及视频编辑质量方面均优于基线方法。