Electroencephalography (EEG) is an objective tool for emotion recognition with promising applications. However, the scarcity of labeled data remains a major challenge in this field, limiting the widespread use of EEG-based emotion recognition. In this paper, a semi-supervised Dual-stream Self-Attentive Adversarial Graph Contrastive learning framework (termed as DS-AGC) is proposed to tackle the challenge of limited labeled data in cross-subject EEG-based emotion recognition. The DS-AGC framework includes two parallel streams for extracting non-structural and structural EEG features. The non-structural stream incorporates a semi-supervised multi-domain adaptation method to alleviate distribution discrepancy among labeled source domain, unlabeled source domain, and unknown target domain. The structural stream develops a graph contrastive learning method to extract effective graph-based feature representation from multiple EEG channels in a semi-supervised manner. Further, a self-attentive fusion module is developed for feature fusion, sample selection, and emotion recognition, which highlights EEG features more relevant to emotions and data samples in the labeled source domain that are closer to the target domain. Extensive experiments conducted on two benchmark databases (SEED and SEED-IV) using a semi-supervised cross-subject leave-one-subject-out cross-validation evaluation scheme show that the proposed model outperforms existing methods under different incomplete label conditions (with an average improvement of 5.83% on SEED and 6.99% on SEED-IV), demonstrating its effectiveness in addressing the label scarcity problem in cross-subject EEG-based emotion recognition.
翻译:脑电图(EEG)是一种客观的情绪识别工具,具有广阔的应用前景。然而,标记数据的稀缺性仍是该领域面临的主要挑战,限制了基于脑电的情绪识别的广泛应用。本文提出了一种半监督双流自注意力对抗图对比学习框架(称为DS-AGC),以应对跨被试脑电情绪识别中标记数据有限的挑战。DS-AGC框架包含两个并行流,分别用于提取非结构化和结构化的脑电特征。非结构化流采用了一种半监督多域自适应方法,以减轻标记源域、未标记源域和未知目标域之间的分布差异。结构化流开发了一种图对比学习方法,以半监督方式从多个脑电通道中提取有效的基于图的特征表示。此外,本文还开发了一个自注意力融合模块,用于特征融合、样本选择和情绪识别,该模块突出强调与情绪更相关的脑电特征,以及标记源域中更接近目标域的数据样本。在两个基准数据库(SEED和SEED-IV)上使用半监督跨被试留一被试交叉验证评估方案进行的广泛实验表明,所提模型在不同不完整标记条件下均优于现有方法(在SEED上平均提升5.83%,在SEED-IV上平均提升6.99%),证明了其在解决跨被试脑电情绪识别中标记稀缺问题方面的有效性。