AI coding agents are increasingly used to generate pull requests (PRs) that propose code fixes in software projects. From a first exploration of the AIDev dataset, we find that 46.41\% of the fixes proposed by the agents Copilot, Devin, Cursor, and Claude are rejected. This represents a significant amount of wasted resources that require human reviews, verifications, and running tests and validations for fixes that are merely discarded. Our goal in this paper is to understand the failure modes of AI-agents, an understanding that is crucial for better integrating AI-agents as efficient teammates. In this paper, we conduct a qualitative study on a representative sample of 306 non-merged pull requests created or co-authored by the agents mentioned earlier, followed by a quantitative analysis of the reasons for rejection. Our qualitative findings identify 14 reasons divided into four high-level categories for rejecting AI-agent fixes. We observe that developers can reject fixes due to fixes whose implementation is incorrect (e.g., incomplete, wrong approach), fixes that do not pass the continuous integration (CI) pipelines and fail tests, fixes for which the agent is unable to perform the implementation (e.g., no code generated, sessions lost), and fixes whose priority is low. Our results shed light on the importance of better guiding the model at these levels: (1) proposing hints about the approach to follow for fixing an issue, (2) outlining constraints or limitations regarding the approaches that should not be taken, and (3) instructing the agent on how to validate the implementation through CI pipelines and without introducing a breaking change. Our results suggest the need for good prioritization of tasks so that generated fixes do not lead to wasted human review efforts or wasted agent resources (e.g., tokens, compute, or allowed number of requests).

翻译：AI编码Agent越来越多地被用于生成拉取请求（PR），以对软件项目提出代码修复。通过对AIDev数据集的初步探索，我们发现由Copilot、Devin、Cursor和Claude等Agent提出的修复中，有46.41%被拒绝。这意味着大量资源被浪费：需要人工审查、验证、运行测试和验证，而这些修复最终却被弃用。本文旨在理解AI Agent的失败模式，这一认知对于更好地将AI Agent整合为高效团队成员至关重要。我们以Agent创建或共同编写的306个未合并拉取请求为代表性样本开展定性研究，随后对拒绝原因进行定量分析。定性研究结果归纳出14条拒绝原因，可分为四大类：实现不正确的修复（如不完整、方法错误）、未能通过持续集成（CI）流水线或测试失败的修复、Agent无法完成实现的修复（如未生成代码、会话丢失），以及优先级低的修复。我们的研究揭示了在以下层面优化模型引导的重要性：（1）针对问题修复方法提出提示性建议，（2）明确不应采用的约束条件或限制，(3) 指导Agent如何通过CI流水线验证实现且不引入破坏性变更。研究结果表明，需对任务进行有效优先级排序，以避免生成的修复浪费人工审查精力或Agent资源（如令牌、计算力或允许的请求次数）。