This paper explores how natural-language descriptions of formal languages can be compared to their formal representations and how semantic differences can be explained. This is motivated from educational scenarios where learners describe a formal language (presented, e.g., by a finite state automaton, regular expression, pushdown automaton, context-free grammar or in set notation) in natural language, and an educational support system has to (1) judge whether the natural-language description accurately describes the formal language, and to (2) provide explanations why descriptions are not accurate. To address this question, we introduce a representation language for formal languages, Nile, which is designed so that Nile expressions can mirror the syntactic structure of natural-language descriptions of formal languages. Nile is sufficiently expressive to cover a broad variety of formal languages, including all regular languages and fragments of context-free languages typically used in educational contexts. Generating Nile expressions that are syntactically close to natural-language descriptions then allows to provide explanations for inaccuracies in the descriptions algorithmically. In experiments on an educational data set, we show that LLMs can translate natural-language descriptions into equivalent, syntactically close Nile expressions with high accuracy - allowing to algorithmically provide explanations for incorrect natural-language descriptions. Our experiments also show that while natural-language descriptions can also be translated into regular expressions (but not context-free grammars), the expressions are often not syntactically close and thus not suitable for providing explanations.

翻译：本文探讨了如何将形式化语言的自然语言描述与其形式化表示进行比较，以及如何解释语义差异。这一研究动机源于教育场景：学习者用自然语言描述一个形式化语言（例如通过有限状态自动机、正则表达式、下推自动机、上下文无关文法或集合表示法呈现），而教育支持系统需要（1）判断自然语言描述是否准确描述了该形式化语言，并（2）对描述不准确的原因提供解释。为应对这一问题，我们引入了一种形式化语言的表示语言Nile，其设计使得Nile表达式能够反映形式化语言自然语言描述的句法结构。Nile具有足够的表达能力，可覆盖广泛的形式化语言，包括所有正则语言以及教育场景中常用的上下文无关语言片段。通过生成句法上接近自然语言描述的Nile表达式，即可算法化地为描述中的不准确之处提供解释。在教育数据集上的实验表明，大语言模型能够以高准确率将自然语言描述翻译为句法结构相近的等效Nile表达式——这使得算法化解释不正确的自然语言描述成为可能。我们的实验同时揭示：虽然自然语言描述也能被翻译为正则表达式（但无法翻译为上下文无关文法），但这些表达式往往在句法上并不接近，因而不适用于提供解释。