Electroencephalography (EEG) is a critical, non-invasive method to monitor electrical brain activity. EEGs can span anywhere from a couple seconds to multiple hours, posing a major hurdle for existing deep learning methods due to two major factors: (1) existing EEG models are predominantly built upon the attention mechanism, incurring quadratic scaling as the sequence length increases, and (2) raw EEG signals must be processed in a sliding-window fashion due to fixed-length input requirements, preventing global understanding of the entire signal. To this extent, we propose CaMBRAIN - the first Causal, Mamba-based state space model (SSM) capable of real-time inference of EEG signals, arguing that bidirectional approaches are needlessly expensive given the causal, unidirectional nature of EEG. However, training such a model is non-trivial, as crucial EEG events can be extremely brief - within fractions of a second - yet separated by long intervals spanning minutes. Current EEG methods use self-supervised objectives that optimize for signal reconstruction, but these are not well suited for streaming SSMs; they fail to explicitly train the hidden state to retain the salient long-range context needed for streaming inference. We therefore introduce a multi-stage self-supervised training pipeline specifically tailored to encourage long-range memory retention and strong performance on EEG signals, while preserving the linear-time complexity of state space models. CaMBRAIN achieves state-of-the-art (SOTA) results across 3 different EEG datasets with >10x higher throughput than existing models, enabling the first model capable of long-range, continuous inference of variable-length EEG signals.

翻译：脑电图（EEG）是一种监测大脑电活动的关键非侵入性方法。EEG信号时长可从数秒跨越至数小时，这为现有深度学习方法带来两大核心挑战：（1）现有EEG模型主要基于注意力机制构建，随着序列长度增加会产生二次方复杂度；（2）由于输入长度固定，原始EEG信号需通过滑动窗口处理，导致无法实现全局信号理解。为此，我们提出CaMBRAIN——首个基于Mamba的因果状态空间模型（SSM），能够实时推断EEG信号。我们认为，鉴于EEG信号的因果单向性，双向方法存在不必要的计算开销。然而，训练此类模型并非易事：关键EEG事件可能极为短暂（不足一秒），却被长达数分钟的长间隔分隔。当前EEG方法使用优化信号重构的自监督目标函数，但这些方法并不适配流式SSM——它们未能显式训练隐藏状态保留流式推理所需的关键长程上下文。因此，我们引入一种专为促进长距离记忆保持与EEG信号强性能而设计的多阶段自监督训练流程，同时保持状态空间模型的线性时间复杂度。CaMBRAIN在三个不同的EEG数据集上均取得最先进（SOTA）结果，吞吐量较现有模型提升10倍以上，成为首个能够对变长EEG信号进行长距离连续推理的模型。