With the increasing demand for video understanding, video moment and highlight detection (MHD) has emerged as a critical research topic. MHD aims to localize all moments and predict clip-wise saliency scores simultaneously. Despite progress made by existing DETR-based methods, we observe that these methods coarsely fuse features from different modalities, which weakens the temporal intra-modal context and results in insufficient cross-modal interaction. To address this issue, we propose MH-DETR (Moment and Highlight Detection Transformer) tailored for MHD. Specifically, we introduce a simple yet efficient pooling operator within the uni-modal encoder to capture global intra-modal context. Moreover, to obtain temporally aligned cross-modal features, we design a plug-and-play cross-modal interaction module between the encoder and decoder, seamlessly integrating visual and textual features. Comprehensive experiments on QVHighlights, Charades-STA, Activity-Net, and TVSum datasets show that MH-DETR outperforms existing state-of-the-art methods, demonstrating its effectiveness and superiority. Our code is available at https://github.com/YoucanBaby/MH-DETR.

翻译：随着视频理解需求的日益增长，视频时刻与高光检测（MHD）已成为一个关键研究课题。MHD旨在同时定位所有时刻并预测片段级别的显著性分数。尽管现有的基于DETR的方法已取得进展，但我们观察到这些方法对不同模态的特征进行了粗糙融合，这弱化了时间维度的模态内上下文，并导致跨模态交互不足。针对此问题，我们提出了专为MHD设计的MH-DETR（时刻与高光检测Transformer）。具体而言，我们在单模态编码器中引入了一种简单高效的池化算子，以捕获全局模态内上下文。此外，为获得时间对齐的跨模态特征，我们在编码器与解码器之间设计了一种即插即用的跨模态交互模块，无缝融合视觉与文本特征。在QVHighlights、Charades-STA、Activity-Net和TVSum数据集上的全面实验表明，MH-DETR优于现有最先进方法，验证了其有效性与优越性。我们的代码已开源至https://github.com/YoucanBaby/MH-DETR。