Affective brain-computer interfaces (aBCIs) are increasingly recognized for their potential in monitoring and interpreting emotional states through electroencephalography (EEG) signals. Current EEG-based emotion recognition methods perform well with short segments of EEG data. However, these methods encounter significant challenges in real-life scenarios where emotional states evolve over extended periods. To address this issue, we propose a Dual Attentive (DuA) transformer framework for long-term continuous EEG emotion analysis. Unlike segment-based approaches, the DuA transformer processes an entire EEG trial as a whole, identifying emotions at the trial level, referred to as trial-based emotion analysis. This framework is designed to adapt to varying signal lengths, providing a substantial advantage over traditional methods. The DuA transformer incorporates three key modules: the spatial-spectral network module, the temporal network module, and the transfer learning module. The spatial-spectral network module simultaneously captures spatial and spectral information from EEG signals, while the temporal network module detects temporal dependencies within long-term EEG data. The transfer learning module enhances the model's adaptability across different subjects and conditions. We extensively evaluate the DuA transformer using a self-constructed long-term EEG emotion database, along with two benchmark EEG emotion databases. On the basis of the trial-based leave-one-subject-out cross-subject cross-validation protocol, our experimental results demonstrate that the proposed DuA transformer significantly outperforms existing methods in long-term continuous EEG emotion analysis, with an average enhancement of 5.28%.
翻译:情感脑机接口(aBCI)通过脑电图(EEG)信号监测和解读情绪状态的潜力日益受到认可。当前基于EEG的情绪识别方法在短片段EEG数据上表现良好。然而,这些方法在现实生活场景中面临显著挑战,因为情绪状态会在较长时间内持续演变。为解决这一问题,我们提出了一种用于长期连续EEG情绪分析的双重注意力(DuA)Transformer框架。与基于片段的方法不同,DuA Transformer将整个EEG试验作为一个整体进行处理,在试验层面识别情绪,这被称为基于试验的情绪分析。该框架旨在适应不同的信号长度,相比传统方法具有显著优势。DuA Transformer包含三个关键模块:空间-频谱网络模块、时序网络模块和迁移学习模块。空间-频谱网络模块同时捕获EEG信号的空间和频谱信息,而时序网络模块则检测长期EEG数据中的时序依赖性。迁移学习模块增强了模型在不同受试者和条件下的适应性。我们使用自建的长期EEG情绪数据库以及两个基准EEG情绪数据库对DuA Transformer进行了广泛评估。基于试验层面的留一受试者跨受试者交叉验证协议,我们的实验结果表明,所提出的DuA Transformer在长期连续EEG情绪分析中显著优于现有方法,平均性能提升达5.28%。