Dense video captioning aims to interpret and describe all temporally localized events throughout an input video. Recent state-of-the-art methods leverage large language models (LLMs) to provide detailed moment descriptions for video data. However, existing VideoLLMs remain challenging in identifying precise event boundaries in untrimmed videos, causing the generated captions to be not properly grounded. In this paper, we propose TA-Prompting, which enhances VideoLLMs via Temporal Anchors that learn to precisely localize events and prompt the VideoLLMs to perform temporal-aware video event understanding. During inference, in order to properly determine the output caption sequence from an arbitrary number of events presented within a video, we introduce an event coherent sampling strategy to select event captions with sufficient coherence across temporal events and cross-modal similarity with the given video. Through extensive experiments on benchmark datasets, we show that our TA-Prompting is favorable against state-of-the-art VideoLLMs, yielding superior performance on dense video captioning and temporal understanding tasks including moment retrieval and temporalQA.

翻译：密集视频描述旨在解释并描述输入视频中所有具有时间定位的事件。当前最先进的方法利用大语言模型为视频数据提供详细的时间片段描述。然而，现有的视频大语言模型在未修剪视频中识别精确事件边界方面仍面临挑战，导致生成的描述未能被恰当地定位。本文提出TA-Prompting方法，该方法通过时序锚点增强视频大语言模型，这些锚点学习精确地定位事件，并提示视频大语言模型执行具有时间感知的视频事件理解。在推理过程中，为了从视频中任意数量的事件中恰当地确定输出描述序列，我们引入了一种事件连贯性采样策略，以选择在时序事件间具有足够连贯性且与给定视频具有跨模态相似性的事件描述。通过在基准数据集上进行大量实验，我们证明TA-Prompting相较于最先进的视频大语言模型具有优势，在密集视频描述及时序理解任务（包括片段检索和时序问答）上均取得了更优的性能。