Continuous electroencephalography (EEG) signals are widely used in affective brain-computer interface (aBCI) applications. However, not all continuously collected EEG signals are relevant or meaningful to the task at hand (e.g., wondering thoughts). On the other hand, manually labeling the relevant parts is nearly impossible due to varying engagement patterns across different tasks and individuals. Therefore, effectively and efficiently identifying the important parts from continuous EEG recordings is crucial for downstream BCI tasks, as it directly impacts the accuracy and reliability of the results. In this paper, we propose a novel unsupervised deep reinforcement learning framework, called Emotion-Agent, to automatically identify relevant and informative emotional moments from continuous EEG signals. Specifically, Emotion-Agent involves unsupervised deep reinforcement learning combined with a heuristic algorithm. We first use the heuristic algorithm to perform an initial global search and form prototype representations of the EEG signals, which facilitates the efficient exploration of the signal space and identify potential regions of interest. Then, we design distribution-prototype reward functions to estimate the interactions between samples and prototypes, ensuring that the identified parts are both relevant and representative of the underlying emotional states. Emotion-Agent is trained using Proximal Policy Optimization (PPO) to achieve stable and efficient convergence. Our experiments compare the performance with and without Emotion-Agent. The results demonstrate that selecting relevant and informative emotional parts before inputting them into downstream tasks enhances the accuracy and reliability of aBCI applications.

翻译：连续脑电图信号在情感脑机接口应用中广泛使用。然而，并非所有连续采集的脑电图信号都与当前任务相关或具有意义。另一方面，由于不同任务和个体间的参与模式存在差异，手动标注相关部分几乎不可行。因此，从连续脑电图记录中有效且高效地识别重要部分对于下游脑机接口任务至关重要，因为它直接影响结果的准确性和可靠性。本文提出了一种名为情感智能体的新型无监督深度强化学习框架，用于从连续脑电图信号中自动识别相关且信息丰富的情感时刻。具体而言，情感智能体结合了无监督深度强化学习与启发式算法。我们首先使用启发式算法进行初始全局搜索，形成脑电图信号的原型表示，这有助于高效探索信号空间并识别潜在的兴趣区域。随后，我们设计了分布原型奖励函数来估计样本与原型之间的相互作用，确保识别出的部分既相关又能代表潜在的情感状态。情感智能体采用近端策略优化算法进行训练，以实现稳定高效的收敛。我们的实验比较了使用与不使用情感智能体的性能差异。结果表明，在将信号输入下游任务前选择相关且信息丰富的情感部分，能够提升情感脑机接口应用的准确性和可靠性。