Code-generating large language models translate natural language into code. However, only a small portion of the infinite space of naturalistic utterances is effective at guiding code generation. For non-expert end-user programmers, learning this is the challenge of abstraction matching. We examine this challenge in the specific context of data analysis in spreadsheets, in a system that maps the users natural language query to Python code using the Codex generator, executes the code, and shows the result. We propose grounded abstraction matching, which bridges the abstraction gap by translating the code back into a systematic and predictable naturalistic utterance. In a between-subjects, think-aloud study (n=24), we compare grounded abstraction matching to an ungrounded alternative based on previously established query framing principles. We find that the grounded approach improves end-users' understanding of the scope and capabilities of the code-generating model, and the kind of language needed to use it effectively.

翻译：代码生成大型语言模型将自然语言转化为代码。然而，在无限的自然语言表达空间中，仅有少量表述能够有效引导代码生成。对于非专业最终用户程序员而言，学习这一能力正是抽象匹配（abstraction matching）的挑战所在。我们在电子表格数据分析的具体场景中，通过一个利用Codex生成器将用户自然语言查询映射为Python代码并执行展示结果的系统，对这一挑战进行了研究。我们提出了一种基于实例的抽象匹配方法（grounded abstraction matching），通过将代码反向翻译为系统化且可预测的自然语言表述，从而弥合抽象鸿沟。在一项受试者间有声思维研究（n=24）中，我们将基于实例的抽象匹配与基于先前查询框架原则的非实例化（ungrounded）方法进行了比较。研究发现，基于实例的方法能提升最终用户对代码生成模型适用范围与能力的理解水平，并帮助他们掌握有效使用该模型所需的语言表达方式。