Artificial Intelligence (AI) increasingly automates various parts of the software development tasks. Although AI has enhanced the productivity of development tasks, it remains unstudied whether AI essentially outperforms humans in type-related programming tasks, such as employing type constructs properly for type safety, during its tasks. Moreover, there is no systematic study that evaluates whether AI agents overuse or misuse the type constructs under the complicated type systems to the same extent as humans. In this study, we present the first empirical analysis to answer these questions in the domain of TypeScript projects. Our findings show that, in contrast to humans, AI agents are 9x more prone to use the 'any' keyword. In addition, we observed that AI agents use advanced type constructs, including those that ignore type checks, more often compared to humans. Surprisingly, even with all these issues, Agentic pull requests (PRs) have 1.8x higher acceptance rates compared to humans for TypeScript. We encourage software developers to carefully confirm the type safety of their codebases whenever they coordinate with AI agents in the development process.
翻译:人工智能(AI)正日益自动化软件开发任务的各个环节。尽管AI提升了开发任务的生产力,但其在类型相关的编程任务(例如在任务过程中为保障类型安全而正确使用类型构造)中是否本质上优于人类,目前仍未得到充分研究。此外,尚无系统性研究评估AI智能体在复杂类型系统下是否与人类同等程度地过度使用或误用类型构造。在本研究中,我们首次通过实证分析在TypeScript项目领域回答了这些问题。我们的研究结果表明,与人类相比,AI智能体使用'any'关键字的倾向性高出9倍。此外,我们观察到AI智能体比人类更频繁地使用高级类型构造,包括那些会绕过类型检查的构造。令人惊讶的是,尽管存在这些问题,在TypeScript项目中,由AI智能体提交的拉取请求(PRs)的接受率仍比人类高出1.8倍。我们鼓励软件开发者在与AI智能体协作开发过程中,务必仔细确认其代码库的类型安全性。