Video moment retrieval and highlight detection have received attention in the current era of video content proliferation, aiming to localize moments and estimate clip relevances based on user-specific queries. Given that the video content is continuous in time, there is often a lack of clear boundaries between temporal events in a video. This boundary ambiguity makes it challenging for the model to learn text-video clip correspondences, resulting in the subpar performance of existing methods in predicting target segments. To alleviate this problem, we propose to solve the two tasks jointly from the perspective of denoising generation. Moreover, the target boundary can be localized clearly by iterative refinement from coarse to fine. Specifically, a novel framework, DiffusionVMR, is proposed to redefine the two tasks as a unified conditional denoising generation process by combining the diffusion model. During training, Gaussian noise is added to corrupt the ground truth, with noisy candidates produced as input. The model is trained to reverse this noise addition process. In the inference phase, DiffusionVMR initiates directly from Gaussian noise and progressively refines the proposals from the noise to the meaningful output. Notably, the proposed DiffusionVMR inherits the advantages of diffusion models that allow for iteratively refined results during inference, enhancing the boundary transition from coarse to fine. Furthermore, the training and inference of DiffusionVMR are decoupled. An arbitrary setting can be used in DiffusionVMR during inference without consistency with the training phase. Extensive experiments conducted on five widely-used benchmarks (i.e., QVHighlight, Charades-STA, TACoS, YouTubeHighlights and TVSum) across two tasks (moment retrieval and/or highlight detection) demonstrate the effectiveness and flexibility of the proposed DiffusionVMR.

翻译：视频时刻检索与高亮检测在当下视频内容激增的时代备受关注，旨在根据用户特定查询定位时刻并评估片段相关性。由于视频内容在时间上具有连续性，时间事件之间往往缺乏清晰边界。这种边界模糊性使得模型难以学习文本-视频片段对应关系，导致现有方法在预测目标片段时性能欠佳。为缓解此问题，本文提出从去噪生成的角度联合求解两个任务，并通过从粗到细的迭代优化实现对目标边界的清晰定位。具体而言，我们提出了一种新型框架DiffusionVMR，通过融合扩散模型将两个任务重新定义为统一的条件去噪生成过程。训练阶段，向真实值添加高斯噪声进行破坏，并生成带噪声候选样本作为输入；模型被训练用于逆向该加噪过程。推理阶段，DiffusionVMR直接从高斯噪声出发，逐步将候选结果从噪声优化至有意义的输出。值得注意的是，所提出的DiffusionVMR继承了扩散模型在推理过程中可迭代优化结果的优点，增强了从粗到细的边界过渡。此外，DiffusionVMR的训练与推理相解耦——推理时可采用任意设置，无需与训练阶段保持一致。在五个广泛使用的基准数据集（即QVHighlight、Charades-STA、TACoS、YouTubeHighlights和TVSum）上针对两个任务（时刻检索和/或高亮检测）开展的大量实验，验证了所提DiffusionVMR的有效性与灵活性。