How can we perform computations over natural language representations to solve tasks that require symbolic and numeric reasoning? We propose natural language embedded programs (NLEP) as a unifying framework for addressing math/symbolic reasoning, natural language understanding, and instruction following tasks. Our approach prompts a language model to generate full Python programs that define functions over data structures which contain natural language representations of structured knowledge. A Python interpreter then executes the generated code and prints the output. Despite using a task-general prompt, we find that this approach can improve upon strong baselines across a range of different tasks including math and symbolic reasoning, text classification, question answering, and instruction following. We further find the generated programs are often interpretable and enable post-hoc verification of the intermediate reasoning steps.

翻译：如何对自然语言表征进行计算，以解决需要符号推理和数值推理的任务？我们提出自然语言嵌入程序（NLEP）作为统一框架，用于处理数学/符号推理、自然语言理解及指令遵循任务。该方法引导语言模型生成完整的Python程序，这些程序在包含结构化知识自然语言表征的数据结构上定义函数，随后由Python解释器执行生成的代码并输出结果。尽管使用任务通用提示，我们发现该方法在数学与符号推理、文本分类、问答及指令遵循等多项任务中均能优于强基线模型。进一步研究表明，生成的程序通常具有可解释性，并能事后验证中间推理步骤。