With the rise of short videos, the demand for selecting appropriate background music (BGM) for a video has increased significantly, video-music retrieval (VMR) task gradually draws much attention by research community. As other cross-modal learning tasks, existing VMR approaches usually attempt to measure the similarity between the video and music in the feature space. However, they (1) neglect the inevitable label noise; (2) neglect to enhance the ability to capture critical video clips. In this paper, we propose a novel saliency-based self-training framework, which is termed SSVMR. Specifically, we first explore to fully make use of the information containing in the training dataset by applying a semi-supervised method to suppress the adverse impact of label noise problem, where a self-training approach is adopted. In addition, we propose to capture the saliency of the video by mixing two videos at span level and preserving the locality of the two original videos. Inspired by back translation in NLP, we also conduct back retrieval to obtain more training data. Experimental results on MVD dataset show that our SSVMR achieves the state-of-the-art performance by a large margin, obtaining a relative improvement of 34.8% over the previous best model in terms of R@1.

翻译：随着短视频的兴起，为视频选取合适背景音乐的需求显著增长，视频-音乐检索任务逐渐受到学术界的广泛关注。与其他跨模态学习任务类似，现有视频-音乐检索方法通常尝试在特征空间中度量视频与音乐的相似性。然而，这些方法（1）忽略了不可避免的标签噪声问题；（2）未能增强对关键视频片段的捕捉能力。本文提出一种新颖的基于显著性的自训练框架，称为SSVMR。具体而言，我们首先探索通过应用半监督方法来抑制标签噪声问题的不利影响，从而充分利用训练数据集中的信息，其中采用了自训练方法。此外，我们提出通过跨跨度混合两个视频并保留两个原始视频的局部性来捕捉视频的显著性。受自然语言处理中回译技术的启发，我们还进行反向检索以获取更多训练数据。在MVD数据集上的实验结果表明，我们的SSVMR方法以显著优势实现了最先进的性能，在R@1指标上较先前最优模型取得了34.8%的相对提升。