Despite the fact that text-to-video (TTV) model has recently achieved remarkable success, there have been few approaches on TTV for its extension to video editing. Motivated by approaches on TTV models adapting from diffusion-based text-to-image (TTI) models, we suggest the video editing framework given only a pretrained TTI model and a single <text, video> pair, which we term Edit-A-Video. The framework consists of two stages: (1) inflating the 2D model into the 3D model by appending temporal modules and tuning on the source video (2) inverting the source video into the noise and editing with target text prompt and attention map injection. Each stage enables the temporal modeling and preservation of semantic attributes of the source video. One of the key challenges for video editing include a background inconsistency problem, where the regions not included for the edit suffer from undesirable and inconsistent temporal alterations. To mitigate this issue, we also introduce a novel mask blending method, termed as sparse-causal blending (SC Blending). We improve previous mask blending methods to reflect the temporal consistency so that the area where the editing is applied exhibits smooth transition while also achieving spatio-temporal consistency of the unedited regions. We present extensive experimental results over various types of text and videos, and demonstrate the superiority of the proposed method compared to baselines in terms of background consistency, text alignment, and video editing quality.

翻译：尽管文本到视频（TTV）模型近期取得了显著成功，但将其扩展至视频编辑领域的方法仍较为有限。受基于扩散的文本到图像（TTI）模型适配至TTV方法的启发，我们提出了仅利用预训练TTI模型和单一<文本，视频>对的视频编辑框架，即Edit-A-Video。该框架包含两个阶段：（1）通过附加时间模块将二维模型扩展为三维模型，并在源视频上进行微调；（2）将源视频反转至噪声空间，并利用目标文本提示和注意力图注入进行编辑。每个阶段分别实现了时间建模与源视频语义属性的保持。视频编辑的关键挑战之一是背景不一致性问题，即未参与编辑的区域会出现不理想且不一致的时间扰动。为缓解此问题，我们提出了一种新颖的掩码融合方法——稀疏因果融合（SC Blending）。该方法改进了先前的掩码融合技术以反映时间一致性，使得编辑区域呈现平滑过渡，同时确保未编辑区域的时空一致性。我们针对多种文本与视频类型开展了大量实验，结果表明所提方法在背景一致性、文本对齐度和视频编辑质量方面均优于基线方法。