Electroencephalography (EEG) signals, known for convenient non-invasive acquisition but low signal-to-noise ratio, have recently gained substantial attention due to the potential to decode natural images. This paper presents a self-supervised framework to demonstrate the feasibility of learning image representations from EEG signals, particularly for object recognition. The framework utilizes image and EEG encoders to extract features from paired image stimuli and EEG responses. Contrastive learning aligns these two modalities by constraining their similarity. With the framework, we attain significantly above-chance results on a comprehensive EEG-image dataset, achieving a top-1 accuracy of 15.6% and a top-5 accuracy of 42.8% in challenging 200-way zero-shot tasks. Moreover, we perform extensive experiments to explore the biological plausibility by resolving the temporal, spatial, spectral, and semantic aspects of EEG signals. Besides, we introduce attention modules to capture spatial correlations, providing implicit evidence of the brain activity perceived from EEG data. These findings yield valuable insights for neural decoding and brain-computer interfaces in real-world scenarios. The code will be released on https://github.com/eeyhsong/NICE-EEG.

翻译：脑电图（EEG）信号以其便捷的非侵入式采集方式著称，但信噪比较低，近来因其解码自然图像的潜力而备受关注。本文提出一种自监督框架，以证明从EEG信号中学习图像表征（特别是用于物体识别）的可行性。该框架利用图像编码器和EEG编码器，从配对图像刺激与EEG响应中提取特征。对比学习通过约束两种模态的相似性对其进行对齐。借助该框架，我们在一个全面的EEG图像数据集上取得了显著高于随机水平的结果，在极具挑战性的200路零样本任务中，top-1准确率达到15.6%，top-5准确率达到42.8%。此外，我们通过解析EEG信号的时域、空域、频域和语义方面，开展了大量实验来探究其生物学合理性。我们还引入注意力模块来捕捉空间相关性，从而为从EEG数据中感知到的脑活动提供隐含证据。这些发现为现实场景中的神经解码和脑机接口提供了宝贵见解。代码将发布于https://github.com/eeyhsong/NICE-EEG。