Grammatical Error Correction (GEC) involves detecting and correcting the wrong usage of grammar. While large language models (LLMs) with in-context learning (ICL) capabilities have shown significant progress on various natural language processing (NLP) tasks, their few-shot performance on GEC remains suboptimal. This is mainly due to the challenge of retrieving suitable in-context demonstrations that capture error patterns instead of semantic similarity. In this paper, we demonstrate that LLMs can inherently capture information related to grammatical errors through their internal states. From these states, we extract the Grammatical Error Representation (GER), an informative and semantically neutral encoding of grammatical errors. Our novel GER-based retrieval method significantly boosts performance in ICL settings on multilingual GEC datasets, improving the precision of correction. For high-resource languages, our results on 8B-sized open-source models match those of closed-source models such as Deepseek2.5 and GPT-4o-mini. For low-resource languages, our $F_{0.5}$ scores surpass the baseline by up to a factor of 1.20. This method provides a more precise and resource-efficient solution for multilingual GEC, offering a promising direction for interpretable GEC research.

翻译：语法错误纠正涉及检测并修正语法错误用法。尽管具备上下文学习能力的大语言模型在各类自然语言处理任务中取得了显著进展，但其在语法错误纠正任务上的少样本表现仍不尽人意。这主要源于现有方法难以检索到能捕捉错误模式而非语义相似性的合适上下文示例。本文证明，大语言模型可通过其内部状态天然捕获与语法错误相关的信息。我们从中提取出语法错误表征——一种信息丰富且语义中立的语法错误编码方式。这种基于语法错误表征的新型检索方法，能显著提升多语言语法错误纠正数据集在上下文学习场景中的性能表现，提高修正精准度。针对高资源语言，我们在8B参数规模的开源模型上取得了与Deepseek2.5、GPT-4o-mini等闭源模型相当的结果。对于低资源语言，我们的F₀.₅分数较基准方法提升最高达1.20倍。该方法为多语言语法错误纠正提供了更精准且资源高效的新方案，为可解释的语法错误纠正研究开辟了具有前景的发展方向。