Deciphering natural language from brain activity through non-invasive devices remains a formidable challenge. Previous non-invasive decoders either require multiple experiments with identical stimuli to pinpoint cortical regions and enhance signal-to-noise ratios in brain activity, or they are limited to discerning basic linguistic elements such as letters and words. We propose a novel approach to decoding continuous language from single-trial non-invasive fMRI recordings, in which a three-dimensional convolutional network augmented with information bottleneck is developed to automatically identify responsive voxels to stimuli, and a character-based decoder is designed for the semantic reconstruction of continuous language characterized by inherent character structures. The resulting decoder can produce intelligible textual sequences that faithfully capture the meaning of perceived speech both within and across subjects, while existing decoders exhibit significantly inferior performance in cross-subject contexts. The ability to decode continuous language from single trials across subjects demonstrates the promising applications of non-invasive language brain-computer interfaces in both healthcare and neuroscience.

翻译：从非侵入式设备中解码大脑活动所反映的自然语言仍是一项艰巨挑战。现有的非侵入式解码器要么需要借助相同刺激的多次实验来定位皮层区域并增强脑活动的信噪比，要么局限于识别字母和单词等基础语言元素。我们提出了一种新方法，可从单次试验的非侵入式功能磁共振成像记录中解码连续语言：开发了一种结合信息瓶颈的三维卷积网络，用于自动识别对刺激有反应的体素；同时设计了一种基于字符的解码器，用于对具有内在字符结构的连续语言进行语义重建。该解码器能生成可理解的文本序列，忠实捕捉被试内及跨被试间感知语音的语义，而现有解码器在跨被试场景下的表现显著较差。从跨被试单次试验中解码连续语言的能力，展现了非侵入式语言脑机接口在医疗健康与神经科学领域的广阔应用前景。