Deciphering the intricacies of the human brain has captivated curiosity for centuries. Recent strides in Brain-Computer Interface (BCI) technology, particularly using motor imagery, have restored motor functions such as reaching, grasping, and walking in paralyzed individuals. However, unraveling natural language from brain signals remains a formidable challenge. Electroencephalography (EEG) is a non-invasive technique used to record electrical activity in the brain by placing electrodes on the scalp. Previous studies of EEG-to-text decoding have achieved high accuracy on small closed vocabularies, but still fall short of high accuracy when dealing with large open vocabularies. We propose a novel method, EEG2TEXT, to improve the accuracy of open vocabulary EEG-to-text decoding. Specifically, EEG2TEXT leverages EEG pre-training to enhance the learning of semantics from EEG signals and proposes a multi-view transformer to model the EEG signal processing by different spatial regions of the brain. Experiments show that EEG2TEXT has superior performance, outperforming the state-of-the-art baseline methods by a large margin of up to 5% in absolute BLEU and ROUGE scores. EEG2TEXT shows great potential for a high-performance open-vocabulary brain-to-text system to facilitate communication.

翻译：解密人脑的复杂性几个世纪以来一直吸引着人类的好奇心。近年来，脑机接口（BCI）技术的进步，特别是利用运动想象，已恢复了瘫痪患者诸如伸手、抓握和行走等运动功能。然而，从脑信号中解析自然语言仍是一项艰巨挑战。脑电图是一种非侵入性技术，通过将电极放置在头皮上来记录大脑的电活动。以往关于脑电解码为文本的研究在小规模封闭词汇集上取得了高准确率，但在处理大规模开放词汇集时仍未能达到高精度。我们提出了一种新方法——EEG2TEXT，旨在提升开放词汇脑电解码的准确性。具体而言，EEG2TEXT利用EEG预训练增强从脑电信号中学习语义的能力，并提出了一个多视图Transformer来建模大脑不同空间区域处理的EEG信号。实验表明，EEG2TEXT具有优越性能，在绝对BLEU和ROUGE分数上比最先进的基线方法高出多达5%。EEG2TEXT展示了构建高性能开放词汇脑到文本系统以促进交流的巨大潜力。