As software engineering moves toward SE3.0, AI agents are increasingly used to carry out development tasks and contribute changes to software projects. It is therefore important to understand the extent of these contributions and how human developers review and intervene, since these factors shape the risks of delegating work to AI agents. While recent studies have examined how AI agents support software development tasks (e.g., code generation, issue resolution, and PR automation), their role in documentation tasks remains underexplored-even though documentation is widely consumed and shapes how developers understand and use software. Using the AIDev, we analyze 1,997 documentation-related pull requests (PRs) authored by AI agents and human developers, where documentation PRs are those that create or modify project documentation artifacts. We find that AI agents submit substantially more documentation-related PRs than humans in the studied repositories. We further observe that agent-authored documentation edits are typically integrated with little follow-up modification from humans, raising concerns about review practices and the reliability of agent-generated documentation. Overall, while AI agents already contribute substantially to documentation workflows, our results suggest concerns for emerging challenges for documentation quality assurance and human-AI collaboration in SE3.0.
翻译:随着软件工程迈向SE3.0,AI智能体正日益被用于执行开发任务并向软件项目贡献变更。因此,理解这些贡献的程度以及人类开发者如何审查和干预至关重要,因为这些因素决定了将工作委托给AI智能体所带来的风险。尽管近期研究已探讨AI智能体如何支持软件开发任务(例如代码生成、问题解决和PR自动化),但它们在文档任务中的作用仍未得到充分探索——尽管文档被广泛使用,并影响着开发者理解和使用软件的方式。利用AIDev,我们分析了由AI智能体和人类开发者撰写的1,997个与文档相关的拉取请求(PRs),其中文档PR指创建或修改项目文档制品的请求。我们发现,在所研究的代码库中,AI智能体提交的文档相关PR数量显著多于人类。我们进一步观察到,智能体撰写的文档编辑通常只需人类极少后续修改即可被集成,这引发了关于审查实践以及智能体生成文档可靠性的担忧。总体而言,虽然AI智能体已在文档工作流中做出重要贡献,但我们的研究结果表明,SE3.0中文档质量保证和人机协作正面临新的挑战。