Few-shot video action recognition is an effective approach to recognizing new categories with only a few labeled examples, thereby reducing the challenges associated with collecting and annotating large-scale video datasets. Existing methods in video action recognition rely on large labeled datasets from the same domain. However, this setup is not realistic as novel categories may come from different data domains that may have different spatial and temporal characteristics. This dissimilarity between the source and target domains can pose a significant challenge, rendering traditional few-shot action recognition techniques ineffective. To address this issue, in this work, we propose a novel cross-domain few-shot video action recognition method that leverages self-supervised learning and curriculum learning to balance the information from the source and target domains. To be particular, our method employs a masked autoencoder-based self-supervised training objective to learn from both source and target data in a self-supervised manner. Then a progressive curriculum balances learning the discriminative information from the source dataset with the generic information learned from the target domain. Initially, our curriculum utilizes supervised learning to learn class discriminative features from the source data. As the training progresses, we transition to learning target-domain-specific features. We propose a progressive curriculum to encourage the emergence of rich features in the target domain based on class discriminative supervised features in the source domain. %a schedule that helps with this transition. We evaluate our method on several challenging benchmark datasets and demonstrate that our approach outperforms existing cross-domain few-shot learning techniques. Our code is available at \hyperlink{https://github.com/Sarinda251/CDFSL-V}{https://github.com/Sarinda251/CDFSL-V}
翻译:小样本视频动作识别是一种有效的方法,仅需少量标注样本即可识别新类别,从而减轻与大规模视频数据集收集和标注相关的挑战。现有视频动作识别方法依赖于同一领域的大规模标注数据集。然而,这种设定并不现实,因为新类别可能来自具有不同时空特性的不同数据领域。源域与目标域之间的这种差异性可能构成重大挑战,导致传统的小样本动作识别技术失效。为解决此问题,本文提出了一种新颖的跨域小样本视频动作识别方法,该方法利用自监督学习和课程学习来平衡源域和目标域的信息。具体而言,我们的方法采用基于掩码自编码器的自监督训练目标,以自监督方式从源数据和目标数据中学习。随后,渐进式课程平衡了从源数据集中学习判别性信息与从目标域中学习通用信息的过程。初始阶段,我们的课程利用监督学习从源数据中学习类别判别性特征;随着训练推进,我们转而学习目标域特定特征。我们提出了一种渐进式课程,基于源域中的类别判别性监督特征,促进目标域中丰富特征的出现。我们在多个具有挑战性的基准数据集上评估了该方法,结果表明我们的方法优于现有的跨域小样本学习技术。我们的代码可在 \hyperlink{https://github.com/Sarinda251/CDFSL-V}{https://github.com/Sarinda251/CDFSL-V} 获取。