Peer review author responses often include commitments to add experiments, release code, or clarify content in the final paper. Yet, there is currently no systematic mechanism to ensure authors fulfill these promises. In this position paper, we present a large-scale audit of author commitments using large language models (LLMs) to compare rebuttals against camera-ready versions. Analyzing the commitments from ICLR-2025 and EMNLP-2024, we find that while a majority of promised changes are implemented, a significant share (about 25%) are not, with "missing experiments" and other high-impact items among the most frequently unfulfilled. We demonstrate that LLM-based tools can feasibly detect the promises. Finally, we propose the idea of Author Commitment Checklist, which would alert authors and organizers to unaddressed promises, increasing accountability and strengthening the integrity of the peer review process. We discuss the benefits of this practice and advocate for its adoption in future conferences.

翻译：同行评审的作者回复中常包含对最终论文增加实验、发布代码或澄清内容的承诺。然而，目前尚无系统性机制确保作者履行这些承诺。在本立场论文中，我们利用大语言模型（LLMs）对作者承诺进行了大规模审计，通过对比驳论与最终版论文。分析ICLR-2025和EMNLP-2024的承诺后发现，尽管大部分承诺变更已实施，但仍有相当比例（约25%）未兑现，其中“缺失实验”及其他高影响力条目是最常未履行的承诺之一。我们证明基于LLM的工具能够可行地检测出这些承诺。最后，我们提出作者承诺核查清单（Author Commitment Checklist）的概念，该清单可提醒作者和组织者未处理的承诺，增强问责机制并强化同行评审过程的完整性。我们讨论了这一实践的益处，并倡导在未来的会议中采用该方法。