This study investigates the real-world impact of the generative AI (GenAI) tool GitHub Copilot on developer activity and perceived productivity. We conducted a mixed-methods case study in NAV IT, a large public sector agile organization. We analyzed 26,317 unique non-merge commits from 703 of NAV IT's GitHub repositories over a two-year period, focusing on commit-based activity metrics from 25 Copilot users and 14 non-users. The analysis was complemented by survey responses on their roles and perceived productivity, as well as 13 interviews. Our analysis of activity metrics revealed that individuals who used Copilot were consistently more active than non-users, even prior to Copilot's introduction. We did not find any statistically significant changes in commit-based activity for Copilot users after they adopted the tool, although minor increases were observed. This suggests a discrepancy between changes in commit-based metrics and the subjective experience of productivity.

翻译：本研究探讨了生成式人工智能工具GitHub Copilot对开发者实际活动与感知生产力的真实影响。我们在挪威劳工与福利管理局信息技术部门开展了一项混合方法案例研究，该机构是一个大型公共部门的敏捷组织。我们分析了来自该机构703个GitHub代码库两年间的26,317个非合并提交记录，重点关注25名Copilot用户和14名非用户的提交活动指标。分析还结合了关于其角色与感知生产力的问卷调查结果以及13次访谈。活动指标分析显示，Copilot用户即使在引入该工具前也始终比非用户更为活跃。尽管观察到轻微增长，但我们未发现Copilot用户在采用该工具后提交活动指标出现统计学显著变化。这表明基于提交的指标变化与生产力主观体验之间存在差异。