As software systems evolve, patches may unintentionally alter program behavior. Validating patches against their intended semantics is difficult due to incomplete regression tests and informal, non-executable natural language (NL) descriptions of patch intent. We present PatchGuru, the first automated technique that infers executable patch specifications from real-world pull requests (PRs). Given a PR, PatchGuru uses large language models (LLMs) to extract developer intent from NL artifacts and synthesizes patch oracles: under-approximate yet practical specifications expressed as runtime assertions in comparison programs that integrate pre- and post-patch versions. Patch oracles focus on patch-relevant behaviors, enable automated validation, and support cross-version properties. PatchGuru iteratively refines inferred oracles by comparing pre- and post-patch behaviors, identifies violations, filters inconsistencies via self-review, and generates bug reports. We evaluate PatchGuru on 400 recent PRs from four widely used open-source Python projects. PatchGuru reports 39 warnings with a precision of 0.62, yielding 24 confirmed true positives, including 12 previously unknown bugs, 11 of which were subsequently fixed by developers. Compared to the state-of-the-art technique Testora, PatchGuru detects 17 more bugs (24 vs. 7) while improving precision from 0.32 to 0.62. PatchGuru incurs an average cost of 8.9 minutes and USD 0.07 per PR. These results suggest that PatchGuru complements code review and regression testing by providing executable documentation and automated validation of patch intent.
翻译:随着软件系统的演进,补丁可能会无意中改变程序行为。由于回归测试的不完整性以及补丁意图的非正式、不可执行的自然语言描述,针对预期语义验证补丁变得十分困难。本文提出PatchGuru,这是首个能从实际拉取请求中推断可执行补丁规约的自动化技术。给定一个拉取请求,PatchGuru利用大语言模型从自然语言制品中提取开发者意图,并合成补丁预言:这是一种以比较程序中的运行时断言形式表达的欠近似但实用的规约,该程序集成了补丁前后的版本。补丁预言聚焦于补丁相关的行为,支持自动化验证,并能处理跨版本属性。PatchGuru通过比较补丁前后的行为迭代优化推断出的预言,识别违规情况,通过自审过滤不一致性,并生成错误报告。我们在四个广泛使用的开源Python项目的400个近期拉取请求上评估PatchGuru。PatchGuru报告了39个警告,精确度为0.62,产生了24个确认的真实正例,其中包括12个先前未知的缺陷,其中11个随后被开发者修复。与最先进技术Testora相比,PatchGuru多检测出17个缺陷(24对7),同时将精确度从0.32提升至0.62。PatchGuru处理每个拉取请求的平均成本为8.9分钟和0.07美元。这些结果表明,PatchGuru通过提供可执行文档和补丁意图的自动化验证,为代码审查和回归测试提供了有效补充。