Due to the ever-increasing complexity of income tax laws in the United States, the number of US taxpayers filing their taxes using tax preparation software (henceforth, tax software) continues to increase. According to the U.S. Internal Revenue Service (IRS), in FY22, nearly 50% of taxpayers filed their individual income taxes using tax software. Given the legal consequences of incorrectly filing taxes for the taxpayer, ensuring the correctness of tax software is of paramount importance. Metamorphic testing has emerged as a leading solution to test and debug legal-critical tax software due to the absence of correctness requirements and trustworthy datasets. The key idea behind metamorphic testing is to express the properties of a system in terms of the relationship between one input and its slightly metamorphosed twinned input. Extracting metamorphic properties from IRS tax publications is a tedious and time-consuming process. As a response, this paper formulates the task of generating metamorphic specifications as a translation task between properties extracted from tax documents - expressed in natural language - to a contrastive first-order logic form. We perform a systematic analysis on the potential and limitations of in-context learning with Large Language Models(LLMs) for this task, and outline a research agenda towards automating the generation of metamorphic specifications for tax preparation software.

翻译：由于美国所得税法日益复杂，使用税务准备软件（以下简称税务软件）报税的美国纳税人数量持续增长。据美国国税局（IRS）统计，2022财年近50%的纳税人通过税务软件申报个人所得税。鉴于错误报税可能给纳税人带来法律后果，确保税务软件的准确性至关重要。蜕变测试因缺乏正确性标准与可信数据集，已成为测试和调试法律关键型税务软件的主流方案。蜕变测试的核心思想是通过原始输入与其轻微变换版本之间的关系来描述系统属性。从IRS税务出版物中提取蜕变属性是一项繁琐耗时的过程。为此，本文将生成蜕变规范的任务形式化为：将税务文档中提取的以自然语言表述的属性，转化为对比一阶逻辑形式的翻译任务。我们系统分析了大型语言模型（LLMs）在上下文学习应用于该任务的潜力与局限，并提出了面向税务准备软件自动化生成蜕变规范的研究路线图。