Electroencephalography (EEG) is a neuroimaging technique that records brain neural activity with high temporal resolution. Unlike other methods, EEG does not require prohibitively expensive equipment and can be easily set up using commercially available portable EEG caps, making it an ideal candidate for brain-computer interfaces. However, EEG signals are characterised by poor spatial resolution and high noise levels, complicating their decoding. In this study, we employ a contrastive learning framework to align encoded EEG features with pretrained CLIP features, achieving a 7% improvement over the state-of-the-art in EEG decoding of object categories. This enhancement is equally attributed to (1) a novel online sampling method that boosts the signal-to-noise ratio and (2) multimodal representations leveraging visual and language features to enhance the alignment space. Our analysis reveals a systematic interaction between the architecture and dataset of pretrained features and their alignment efficacy for EEG signal decoding. This interaction correlates with the generalisation power of the pretrained features on ImageNet-O/A datasets ($r=.5$). These findings extend beyond EEG signal alignment, offering potential for broader applications in neuroimaging decoding and generic feature alignments.

翻译：脑电图（EEG）是一种具有高时间分辨率的神经影像技术，用于记录大脑神经活动。与其他方法不同，EEG无需昂贵的设备，使用市售便携式EEG帽即可轻松搭建，使其成为脑机接口的理想选择。然而，EEG信号具有空间分辨率低、噪声水平高的特点，这使其解码变得复杂。在本研究中，我们采用对比学习框架，将编码后的EEG特征与预训练的CLIP特征对齐，在物体类别的EEG解码任务中实现了相对于现有最佳性能7%的提升。这一改进同样归功于：（1）一种新颖的在线采样方法，提高了信噪比；（2）利用视觉和语言特征的多模态表示，以增强对齐空间。我们的分析揭示了预训练特征的架构、数据集与其在EEG信号解码中对齐效果之间的系统性相互作用。这种相互作用与预训练特征在ImageNet-O/A数据集上的泛化能力相关（$r=.5$）。这些发现不仅适用于EEG信号对齐，还为神经影像解码和通用特征对齐的更广泛应用提供了潜力。