Weakly supervised Audio-Visual Video Parsing (AVVP) aims to recognize and temporally localize audio, visual, and audio-visual events in videos using only coarse-grained labels. Faced with the challenging task settings, existing research advances along two main paths: pre-training pseudo-label generators for fine-grained cross-modal semantic guidance, or refining AVVP model architectures to enhance audio-visual fusion. However, since audio and visual signals are typically unaligned, achieving accurate video parsing fundamentally relies on precise perception of uni-modal events. Yet these multi-modal focused strategies excessively emphasize multi-modal fusion while inadequately guiding and preserving uni-modal semantics, resulting in noisy pseudo-labels and sub-optimal video parsing performance. This paper proposes a novel framework that enhances uni-modal representations for both the pseudo-label generator and the AVVP model. Specifically, we introduce a similarity-based label migration approach to annotate pre-training data, thereby enabling the pseudo-label generator to better understand uni-modal events. We also employ a soft-constrained manner to refine modeling of uni-modal features in parallel with multi-modal fusion. These designs enable coordinated attention to both uni-modal and cross-modal representations, thus boosting the localization performance for events. Extensive experiments show that our method outperforms state-of-the-art methods in both pseudo-label and AVVP performance.

翻译：弱监督音视频解析旨在仅利用粗粒度标签，识别并时间定位视频中的音频、视觉及音视频事件。面对这一具有挑战性的任务设定，现有研究沿两条主要路径推进：预训练用于细粒度跨模态语义引导的伪标签生成器，或改进音视频解析模型架构以增强音视频融合。然而，由于音频和视觉信号通常不对齐，实现精确的视频解析从根本上依赖于对单模态事件的准确感知。但这些以多模态为核心的策略过度强调融合，却未能充分引导和保留单模态语义，导致伪标签噪声大且视频解析性能次优。本文提出了一种新颖框架，同时增强伪标签生成器和音视频解析模型的单模态表示。具体而言，我们引入基于相似度的标签迁移方法对预训练数据进行标注，使伪标签生成器能更好地理解单模态事件；同时采用软约束方式，在与多模态融合并行中细化单模态特征建模。这些设计使模型能协同关注单模态与跨模态表征，从而提升事件定位性能。大量实验表明，本方法在伪标签生成与音视频解析任务上均优于现有最优方法。