Recent large-scale video datasets have facilitated the generation of diverse open-domain videos of Video Diffusion Models (VDMs). Nonetheless, the efficacy of VDMs in assimilating complex knowledge from these datasets remains constrained by their inherent scale, leading to suboptimal comprehension and synthesis of numerous actions. In this paper, we introduce EchoReel, a novel approach to augment the capability of VDMs in generating intricate actions by emulating motions from pre-existing videos, which are readily accessible from databases or online repositories. EchoReel seamlessly integrates with existing VDMs, enhancing their ability to produce realistic motions without compromising their fundamental capabilities. Specifically, the Action Prism (AP), is introduced to distill motion information from reference videos, which requires training on only a small dataset. Leveraging the knowledge from pre-trained VDMs, EchoReel incorporates new action features into VDMs through the additional layers, eliminating the need for any further fine-tuning of untrained actions. Extensive experiments demonstrate that EchoReel is not merely replicating the whole content from references, and it significantly improves the generation of realistic actions, even in situations where existing VDMs might directly fail.

翻译：近期的大规模视频数据集促进了视频扩散模型（VDMs）生成多样化的开放域视频。然而，VDMs在吸收这些数据集中的复杂知识方面仍受限于其固有规模，导致对众多动作的理解与合成效果欠佳。本文提出EchoReel，一种通过模拟已有视频（可从数据库或在线存储库中轻易获取）中的运动来增强VDMs生成复杂动作能力的新方法。EchoReel可与现有VDMs无缝集成，在不损害其基础能力的前提下提升其生成逼真运动的效果。具体而言，我们引入了动作棱镜（AP）以从参考视频中提取运动信息，该模块仅需在小规模数据集上训练。借助预训练VDMs的知识，EchoReel通过新增层将动作特征融入VDMs，从而无需对未训练的动作进行任何额外微调。大量实验证明，EchoReel并非简单复制参考视频的全部内容，而是显著提升了逼真动作的生成能力，甚至在现有VDMs可能直接失效的场景下依然有效。