Decoding the human brain has been a hallmark of neuroscientists and Artificial Intelligence researchers alike. Reconstruction of visual images from brain Electroencephalography (EEG) signals has garnered a lot of interest due to its applications in brain-computer interfacing. This study proposes a two-stage method where the first step is to obtain EEG-derived features for robust learning of deep representations and subsequently utilize the learned representation for image generation and classification. We demonstrate the generalizability of our feature extraction pipeline across three different datasets using deep-learning architectures with supervised and contrastive learning methods. We have performed the zero-shot EEG classification task to support the generalizability claim further. We observed that a subject invariant linearly separable visual representation was learned using EEG data alone in an unimodal setting that gives better k-means accuracy as compared to a joint representation learning between EEG and images. Finally, we propose a novel framework to transform unseen images into the EEG space and reconstruct them with approximation, showcasing the potential for image reconstruction from EEG signals. Our proposed image synthesis method from EEG shows 62.9% and 36.13% inception score improvement on the EEGCVPR40 and the Thoughtviz datasets, which is better than state-of-the-art performance in GAN.

翻译：解码人脑一直是神经科学家和人工智能研究人员的重要目标。从脑电图信号重建视觉图像因其在脑机接口中的应用而备受关注。本研究提出一种两阶段方法：第一步获取基于脑电的特征以鲁棒地学习深层表征，随后利用所学表征进行图像生成与分类。我们通过采用监督学习和对比学习方法的深度学习架构，在三个不同数据集上证明了特征提取管道的泛化能力。为进一步支持泛化性主张，我们执行了零样本脑电分类任务。观察到在单模态设置下仅利用脑电数据即可学习到受试者不变且线性可分的视觉表征，其k-means聚类准确率优于脑电与图像的联合表征学习。最后，我们提出一种将未见图像转换至脑电空间并近似重建的新框架，展示了从脑电信号进行图像重建的潜力。我们提出的脑电图像合成方法在EEGCVPR40和Thoughtviz数据集上分别实现了62.9%和36.13%的初始分数提升，优于当前生成对抗网络的最优性能。