In this research, we present a novel approach to motion customization in video generation, addressing the widespread gap in the thorough exploration of motion representation within video generative models. Recognizing the unique challenges posed by video's spatiotemporal nature, our method introduces Motion Embeddings, a set of explicit, temporally coherent one-dimensional embeddings derived from a given video. These embeddings are designed to integrate seamlessly with the temporal transformer modules of video diffusion models, modulating self-attention computations across frames without compromising spatial integrity. Our approach offers a compact and efficient solution to motion representation and enables complex manipulations of motion characteristics through vector arithmetic in the embedding space. Furthermore, we identify the Temporal Discrepancy in video generative models, which refers to variations in how different motion modules process temporal relationships between frames. We leverage this understanding to optimize the integration of our motion embeddings. Our contributions include the introduction of a tailored motion embedding for customization tasks, insights into the temporal processing differences in video models, and a demonstration of the practical advantages and effectiveness of our method through extensive experiments.

翻译：本研究提出一种面向视频生成的运动定制新方法，系统性地解决了视频生成模型中运动表征探索不足的普遍问题。针对视频时空特性带来的独特挑战，本方法创新性地引入了运动嵌入（Motion Embeddings）——一组从给定视频中提取的显式、时间连贯的一维嵌入。这些嵌入可无缝集成至视频扩散模型的时间变换器模块，通过跨帧调节自注意力计算，且不影响空间完整性。本方案不仅提供了紧凑高效的运动表征方案，还能通过嵌入空间中的向量算术实现运动特征的复杂操控。此外，我们识别出视频生成模型中的时间差异（Temporal Discrepancy），即不同运动模块在处理帧间时序关系时存在的差异性。基于此认知，我们优化了运动嵌入的集成方案。主要贡献包括：面向定制化任务设计的定制运动嵌入、对视频模型时序处理差异性的深刻洞见，以及通过大量实验验证方法实用优势与有效性的实证成果。