Continuous electroencephalography (EEG) is routinely used in neurocritical care to monitor seizures and other harmful brain activity, including rhythmic and periodic patterns that are clinically significant. Although deep learning methods have achieved high accuracy in seizure detection, most existing approaches remain seizure-centric, rely on discrete-label supervision, and are primarily evaluated using accuracy-based metrics. A central limitation of current EEG modeling practice is the weak correspondence between learned representations and how EEG findings are interpreted and summarized in clinical workflows. Harmful EEG activity exhibits overlapping patterns, graded expert agreement, and temporal persistence, which are not well captured by classification objectives alone. This work proposes a multimodal EEG representation learning framework that integrates signal-domain modeling with structured clinical language supervision. First, raw EEG is transformed into a longitudinal bipolar montage and time-frequency representations. Second, dual transformer-based encoders model complementary temporal and frequency-centric dependencies and are fused using an adaptive gating mechanism. Third, EEG embeddings are aligned with structured expert consensus descriptions through a contrastive objective. Finally, an EEG-conditioned text reconstruction loss is introduced as a representation-level constraint alongside standard classification loss. Experimental evaluation using a controlled train-validation-test split achieves a six-class test accuracy of 0.9797. Ablation analyses show that removing contrastive alignment reduces cross-modal retrieval performance from Recall@10 of 0.3390 to 0.0045, despite minimal change in classification accuracy. These findings demonstrate that discriminative accuracy does not reliably reflect representation quality for clinically meaningful EEG modeling.

翻译：连续脑电图（EEG）在神经重症监护中被常规用于监测癫痫发作及其他有害脑活动，包括具有临床意义的节律性和周期性模式。尽管深度学习方法在癫痫检测方面已取得高准确率，但现有方法大多仍以癫痫为中心，依赖于离散标签监督，且主要使用基于准确率的指标进行评估。当前EEG建模实践的一个核心局限在于，学习到的表征与临床工作流程中EEG结果的解读和总结方式之间对应关系较弱。有害的EEG活动呈现出重叠的模式、分级的专家共识以及时间持续性，这些特性仅靠分类目标难以充分捕捉。本研究提出了一种多模态EEG表征学习框架，将信号域建模与结构化临床语言监督相结合。首先，原始EEG被转换为纵向双极导联和时频表示。其次，基于双Transformer的编码器对互补的时域和频域依赖关系进行建模，并通过自适应门控机制进行融合。第三，EEG嵌入通过对比学习目标与结构化专家共识描述进行对齐。最后，在标准分类损失之外，引入了一种以EEG为条件的文本重建损失作为表征层面的约束。使用受控的训练-验证-测试划分进行的实验评估实现了六分类测试准确率0.9797。消融分析表明，移除对比对齐会使跨模态检索性能从Recall@10为0.3390降至0.0045，尽管分类准确率变化极小。这些发现证明，对于具有临床意义的EEG建模，判别准确率并不能可靠地表征其表征质量。