Deciphering natural language from brain activity through non-invasive devices remains a formidable challenge. Previous non-invasive decoders either require multiple experiments with identical stimuli to pinpoint cortical regions and enhance signal-to-noise ratios in brain activity, or they are limited to discerning basic linguistic elements such as letters and words. We propose a novel approach to decoding continuous language from single-trial non-invasive fMRI recordings, in which a three-dimensional convolutional network augmented with information bottleneck is developed to automatically identify responsive voxels to stimuli, and a character-based decoder is designed for the semantic reconstruction of continuous language characterized by inherent character structures. The resulting decoder can produce intelligible textual sequences that faithfully capture the meaning of perceived speech both within and across subjects, while existing decoders exhibit significantly inferior performance in cross-subject contexts. The ability to decode continuous language from single trials across subjects demonstrates the promising applications of non-invasive language brain-computer interfaces in both healthcare and neuroscience.

翻译：通过非侵入式设备从脑部活动中解码自然语言仍是一项艰巨挑战。以往的非侵入式解码器要么需要多次使用相同刺激的实验来精确定位皮层区域并提高脑部活动的信噪比，要么仅能辨别字母和单词等基本语言元素。我们提出一种新颖方法，通过单次试验的非侵入式fMRI记录来解码连续语言：该方法开发了一种结合信息瓶颈的三维卷积网络，可自动识别对刺激有反应的体素；同时设计了一种基于字符的解码器，用于对具有固有字符结构的连续语言进行语义重建。由此产生的解码器能够生成可理解的文本序列，准确捕捉被试内及跨被试感知言语的含义，而现有解码器在跨被试场景中的表现则显著逊色。从跨被试的单次试验中解码连续语言的能力，彰显了非侵入式语言脑机接口在医疗保健与神经科学领域的广阔应用前景。