Electroencephalography (EEG) is a non-invasive technique to measure and record brain electrical activity, widely used in various BCI and healthcare applications. Early EEG decoding methods rely on supervised learning, limited by specific tasks and datasets, hindering model performance and generalizability. With the success of large language models, there is a growing body of studies focusing on EEG foundation models. However, these studies still leave challenges: Firstly, most of existing EEG foundation models employ full EEG modeling strategy. It models the spatial and temporal dependencies between all EEG patches together, but ignores that the spatial and temporal dependencies are heterogeneous due to the unique structural characteristics of EEG signals. Secondly, existing EEG foundation models have limited generalizability on a wide range of downstream BCI tasks due to varying formats of EEG data, making it challenging to adapt to. To address these challenges, we propose a novel foundation model called CBraMod. Specifically, we devise a criss-cross transformer as the backbone to thoroughly leverage the structural characteristics of EEG signals, which can model spatial and temporal dependencies separately through two parallel attention mechanisms. And we utilize an asymmetric conditional positional encoding scheme which can encode positional information of EEG patches and be easily adapted to the EEG with diverse formats. CBraMod is pre-trained on a very large corpus of EEG through patch-based masked EEG reconstruction. We evaluate CBraMod on up to 10 downstream BCI tasks (12 public datasets). CBraMod achieves the state-of-the-art performance across the wide range of tasks, proving its strong capability and generalizability. The source code is publicly available at \url{https://github.com/wjq-learning/CBraMod}.

翻译：脑电图（EEG）是一种测量和记录脑电活动的非侵入性技术，广泛应用于各类脑机接口（BCI）与医疗健康领域。早期的脑电解码方法依赖于监督学习，受限于特定任务与数据集，制约了模型性能与泛化能力。随着大语言模型取得的成功，针对脑电基础模型的研究日益增多。然而，现有研究仍面临挑战：首先，多数现有脑电基础模型采用全脑电建模策略，即对所有脑电片段的空间与时间依赖性进行联合建模，但忽略了由于脑电信号独特的结构特性，其空间依赖性与时间依赖性本质上是异质的。其次，现有脑电基础模型因脑电数据格式多样而难以适配，导致其在广泛的下游BCI任务上泛化能力有限。为应对这些挑战，我们提出了一种名为CBraMod的新型基础模型。具体而言，我们设计了一种交叉Transformer作为主干网络，以充分利用脑电信号的结构特性，该网络通过两个并行的注意力机制分别建模空间与时间依赖性。同时，我们采用了一种非对称条件位置编码方案，该方案能够编码脑电片段的位置信息，并可轻松适配不同格式的脑电数据。CBraMod通过基于片段的掩码脑电重建任务，在超大规模脑电语料上进行预训练。我们在多达10个下游BCI任务（涵盖12个公开数据集）上评估CBraMod。实验结果表明，CBraMod在广泛任务中均取得了最先进的性能，证明了其强大的能力与泛化性。源代码已公开于 \url{https://github.com/wjq-learning/CBraMod}。