Decoding natural language from non-invasive EEG signals is a promising yet challenging task. However, current state-of-the-art models remain constrained by three fundamental issues: Semantic Bias, where outputs collapse into generic linguistic templates; Signal Neglect, where models rely heavily on LLM priors to hallucinate fluent text even in the absence of meaningful signals; and the "BLEU Trap", where high-frequency stopwords inflate n-gram metrics, masking a lack of true semantic fidelity. To resolve these challenges, we move beyond conventional end-to-end pipelines and propose SemKey, a novel multi-stage framework that enforces signal-grounded generation through four decoupled semantic objectives: sentiment, topic, length, and surprisal. We extract these semantic anchors from EEG embeddings directly, then unify them with an Active Retrieval Decoding mechanism, compelling the LLM to ground its token generation in the neural signals rather than defaulting to linguistic priors. Furthermore, we break the BLEU Trap by establishing a comprehensive evaluation protocol using rigorous retrieval and distribution-based metrics such as Fréchet Distance. Extensive experiments demonstrate that SemKey effectively mitigates hallucinations on noise inputs and achieves SOTA performance on these robust protocols. Code will be released upon acceptance at https://github.com/xmed-lab/SemKey.

翻译：从非侵入性脑电信号解码自然语言是一项充满前景但极具挑战的任务。然而，当前最先进的模型仍受制于三个根本性问题：语义偏差导致输出坍缩为通用语言模板；信号忽视使模型过度依赖大语言模型先验，即便缺乏有效信号也能生成流畅文本；以及"BLEU陷阱"——高频停用词虚增n-gram指标，掩盖真实语义保真度的缺失。为应对这些挑战，我们突破传统端到端流水线，提出SemKey——一种通过四种解耦语义目标（情感、主题、长度及意外度）强制实现信号锚定生成的新型多阶段框架。我们直接从脑电图嵌入中提取这些语义锚点，并借助主动检索解码机制将其统一，迫使大语言模型在神经信号基础上进行令牌生成，而非默认采用语言先验。此外，我们通过严谨的检索指标及基于分布的评估指标（如弗雷歇距离）构建综合评估协议，从而打破BLEU陷阱。大量实验表明，SemKey有效抑制了噪声输入下的幻觉现象，并在这些稳健协议上达到了最优性能。代码将在论文接收后于https://github.com/xmed-lab/SemKey开放。