Decoding language from brain dynamics is an important open direction in the realm of brain-computer interface (BCI), especially considering the rapid growth of large language models. Compared to invasive-based signals which require electrode implantation surgery, non-invasive neural signals (e.g. EEG, MEG) have attracted increasing attention considering their safety and generality. However, the exploration is not adequate in three aspects: 1) previous methods mainly focus on EEG but none of the previous works address this problem on MEG with better signal quality; 2) prior works have predominantly used $``teacher-forcing"$ during generative decoding, which is impractical; 3) prior works are mostly $``BART-based"$ not fully auto-regressive, which performs better in other sequence tasks. In this paper, we explore the brain-to-text translation of MEG signals in a speech-decoding formation. Here we are the first to investigate a cross-attention-based ``whisper" model for generating text directly from MEG signals without teacher forcing. Our model achieves impressive BLEU-1 scores of 60.30 and 52.89 without pretraining $\&$ teacher-forcing on two major datasets ($\textit{GWilliams}$ and $\textit{Schoffelen}$). This paper conducts a comprehensive review to understand how speech decoding formation performs on the neural decoding tasks, including pretraining initialization, training $\&$ evaluation set splitting, augmentation, and scaling law. Code is available at https://github.com/NeuSpeech/NeuSpeech1$.

翻译：从大脑动态活动中解码语言是脑机接口（BCI）领域一个重要的开放方向，尤其是在大语言模型快速发展的背景下。与需要植入电极手术的侵入式信号相比，非侵入式神经信号（如脑电图EEG、脑磁图MEG）因其安全性和普适性而受到越来越多的关注。然而，现有研究在以下三个方面尚不充分：1）先前方法主要集中于EEG，尚无工作在信号质量更优的MEG上解决此问题；2）先前工作在生成式解码中主要使用“教师强制”策略，这在实际中并不可行；3）先前工作大多基于“BART”架构，并非完全自回归，而在其他序列任务中完全自回归模型通常表现更佳。本文以语音解码的形式，探索MEG信号的脑到文本翻译。我们首次研究了一种基于交叉注意力的“whisper”模型，用于直接从MEG信号生成文本，且无需教师强制。在两个主要数据集（$\textit{GWilliams}$ 和 $\textit{Schoffelen}$）上，我们的模型在未进行预训练且不使用教师强制的条件下，取得了令人印象深刻的BLEU-1分数，分别为60.30和52.89。本文进行了全面分析，以理解语音解码架构在神经解码任务上的表现，包括预训练初始化、训练与评估集划分、数据增强以及缩放规律。代码发布于 https://github.com/NeuSpeech/NeuSpeech1$。