The increasing popularity of AI, particularly Large Language Models (LLMs), has significantly impacted various domains, including Software Engineering. This study explores the integration of AI tools in software engineering practices within a large organization. We focus on ANZ Bank, which employs over 5000 engineers covering all aspects of the software development life cycle. This paper details an experiment conducted using GitHub Copilot, a notable AI tool, within a controlled environment to evaluate its effectiveness in real-world engineering tasks. Additionally, this paper shares initial findings on the productivity improvements observed after GitHub Copilot was adopted on a large scale, with about 1000 engineers using it. ANZ Bank's six-week experiment with GitHub Copilot included two weeks of preparation and four weeks of active testing. The study evaluated participant sentiment and the tool's impact on productivity, code quality, and security. Initially, participants used GitHub Copilot for proposed use-cases, with their feedback gathered through regular surveys. In the second phase, they were divided into Control and Copilot groups, each tackling the same Python challenges, and their experiences were again surveyed. Results showed a notable boost in productivity and code quality with GitHub Copilot, though its impact on code security remained inconclusive. Participant responses were overall positive, confirming GitHub Copilot's effectiveness in large-scale software engineering environments. Early data from 1000 engineers also indicated a significant increase in productivity and job satisfaction.

翻译：人工智能，特别是大型语言模型（LLMs）的日益普及，已显著影响包括软件工程在内的各个领域。本研究探讨了大型组织中AI工具在软件工程实践中的整合情况。我们聚焦于澳新银行（ANZ Bank），该银行拥有超过5000名工程师，涵盖软件开发生命周期的各个方面。本文详细介绍了在一个受控环境中使用GitHub Copilot（一款知名AI工具）进行的实验，以评估其在实际工程任务中的有效性。此外，本文分享了在大规模采用GitHub Copilot后观察到的生产力提升的初步发现，约有1000名工程师使用了该工具。澳新银行为期六周的GitHub Copilot实验包括两周的准备期和四周的积极测试期。研究评估了参与者的情绪以及该工具对生产力、代码质量和安全性的影响。初始阶段，参与者使用GitHub Copilot处理提议的用例，并通过定期调查收集反馈。在第二阶段，他们被分为对照组和Copilot组，每组处理相同的Python挑战，并再次调查其体验。结果显示，使用GitHub Copilot显著提升了生产力和代码质量，但其对代码安全性的影响尚无定论。参与者反馈总体积极，证实了GitHub Copilot在大型软件工程环境中的有效性。来自1000名工程师的早期数据也表明，生产力和工作满意度显著提升。