Current multimodal large language models (MLLMs) have demonstrated remarkable capabilities in short-form video understanding, yet translating long-form cinematic videos into detailed, temporally grounded scripts remains a significant challenge. This paper introduces the novel video-to-script (V2S) task, aiming to generate hierarchical, scene-by-scene scripts encompassing character actions, dialogues, expressions, and audio cues. To facilitate this, we construct a first-of-its-kind human-annotated benchmark and propose a temporally-aware hierarchical evaluation framework. Furthermore, we present OmniScript, an 8B-parameter omni-modal (audio-visual) language model tailored for long-form narrative comprehension. OmniScript is trained via a progressive pipeline that leverages chain-of-thought supervised fine-tuning for plot and character reasoning, followed by reinforcement learning using temporally segmented rewards. Extensive experiments demonstrate that despite its parameter efficiency, OmniScript significantly outperforms larger open-source models and achieves performance comparable to state-of-the-art proprietary models, including Gemini 3-Pro, in both temporal localization and multi-field semantic accuracy.
翻译:当前多模态大语言模型在短视频理解任务中展现出卓越能力,但将长片电影转化为具备精细时间标注的结构化剧本仍是一项重大挑战。本文提出新型视频转剧本任务,旨在生成包含角色动作、对话、表情及音频线索的分层逐场剧本。为此,我们构建首个基于人工标注的基准数据集,并提出一种时间感知的分层评估框架。此外,我们提出全影8B参数全模态(音视频)语言模型,专为长片叙事理解设计。该模型采用渐进式训练流程:先通过思维链监督微调实现情节与角色推理,再结合时间分段奖励的强化学习进行优化。大量实验表明,尽管参数量精简,全影在时间定位与多领域语义准确性方面显著超越更大规模的开源模型,并达到与包括Gemini 3-Pro在内的顶尖闭源模型相当的性能。