In this paper, we present a comparative study of five autonomous coding agents using AIDev-pop, which is a public dataset containing thousands of AI-generated pull requests (PRs) across popular open-source repositories. We evaluate agents' performance along three task-aware dimensions spanning the PR lifecycle: (1) PR acceptance rate, (2) review discussion volume, and (3) commit message quality. Our quantitative analysis finds that Codex consistently achieves high PR acceptance rates across most task categories, while Copilot's PRs trigger the highest volume of both human and automated review discussions. In contrast, commit-level quality varies independently of acceptance outcomes. Claude and Cursor produce higher proportions of high-quality commit messages across several task types, and Codex exhibiting comparatively lower commit quality despite strong integration outcomes. Our findings inform selection and improvements of AI agents for their effective integration to collaborative software engineering.

翻译：本文基于AIDev-pop公共数据集，对五种自主编程智能体进行了比较研究。该数据集包含数千个跨热门开源仓库的AI生成拉取请求（PR）。我们沿PR生命周期的三个任务感知维度评估智能体性能：（1）PR接受率，（2）评审讨论量，以及（3）提交信息质量。定量分析表明：Codex在多数任务类别中始终保持较高的PR接受率；Copilot生成的PR则引发了最多的人工与自动化评审讨论。相比之下，提交级质量与接受结果呈现独立变化趋势：Claude和Cursor在多个任务类型中产生高质量提交信息的比例更高，而Codex尽管集成结果优异，其提交质量却相对较低。本研究为AI智能体的选择与改进提供了依据，以促进其在协同软件工程中的有效集成。