Developers increasingly use AI tools such as ChatGPT, Copilot, and Claude in everyday software workflows, but prior studies often evaluate LLM outputs in isolation rather than examining how developers adapt them in real projects. We analyze 35,361 GitHub code comments that explicitly reference AI use and their associated code blocks. We first open-code 500 unique comments and code blocks to derive a taxonomy of AI-assisted development activities, then annotate the full dataset using two LLM-based classifiers and aggregate predictions with Dawid-Skene expectation-maximization. We also analyze 12,996 subsequent commit messages to study how AI-assisted code evolves after introduction, and examine temporal trends from December 2022 to March 2026. Our results show that developers primarily use LLMs for code implementation, followed by code enhancement, debugging, documentation, and testing. Subsequent commits frequently involve refactoring and cleanup, feature integration and extension, and bug fixing, indicating sustained human oversight in adapting AI-assisted code. Over time, AI-referencing comments shift from direct code generation toward knowledge and conceptual support and code enhancement. These findings suggest that AI tools are becoming embedded not only as code-generation aids, but also as collaborative support mechanisms whose outputs are refined, extended, and corrected by developers over time.
翻译:开发者越来越多地使用ChatGPT、Copilot和Claude等AI工具处理日常软件工作流,但现有研究往往孤立评估大语言模型输出,而非考察开发者如何在真实项目中调整这些输出。我们分析了35,361条明确引用AI使用的GitHub代码注释及其关联的代码块。首先对500条独特注释与代码块进行开放编码,推导出AI辅助开发活动分类体系,随后使用两个基于LLM的分类器对完整数据集进行标注,并通过Dawid-Skene期望最大化算法聚合预测结果。我们还分析了12,996条后续提交信息,以研究AI辅助代码在引入后的演化过程,并考察了2022年12月至2026年3月的时间趋势。结果表明,开发者主要使用LLM进行代码实现,其次是代码增强、调试、文档编写和测试。后续提交频繁涉及重构与清理、功能集成与扩展、以及错误修复,表明在适配AI辅助代码过程中存在持续的人工监管。随时间推移,引用AI的注释从直接代码生成转向知识概念支持和代码增强。这些发现表明,AI工具不仅作为代码生成辅助手段,更逐渐嵌入为协作支持机制,其输出经开发者持续修正、扩展与完善。