We introduce $\textit{InteractiveVideo}$, a user-centric framework for video generation. Different from traditional generative approaches that operate based on user-provided images or text, our framework is designed for dynamic interaction, allowing users to instruct the generative model through various intuitive mechanisms during the whole generation process, e.g. text and image prompts, painting, drag-and-drop, etc. We propose a Synergistic Multimodal Instruction mechanism, designed to seamlessly integrate users' multimodal instructions into generative models, thus facilitating a cooperative and responsive interaction between user inputs and the generative process. This approach enables iterative and fine-grained refinement of the generation result through precise and effective user instructions. With $\textit{InteractiveVideo}$, users are given the flexibility to meticulously tailor key aspects of a video. They can paint the reference image, edit semantics, and adjust video motions until their requirements are fully met. Code, models, and demo are available at https://github.com/invictus717/InteractiveVideo
翻译:我们提出$\textit{InteractiveVideo}$,一种以用户为中心的视频生成框架。与基于用户提供图像或文本的传统生成方法不同,该框架专为动态交互设计,允许用户在生成全过程中通过多种直观机制(如文本与图像提示、绘制、拖放等)指导生成模型。我们提出一种协同多模态指令机制,旨在将用户的多模态指令无缝集成至生成模型中,从而促进用户输入与生成过程之间的协作响应式交互。该方法通过精准有效的用户指令,实现对生成结果的迭代式细粒度优化。借助$\textit{InteractiveVideo}$,用户可灵活定制视频的关键属性:绘制参考图像、编辑语义内容、调整运动模式,直至完全满足需求。代码、模型及演示已发布于https://github.com/invictus717/InteractiveVideo。