Agentic AI coding tools write code with increasing autonomy and in doing so decide when to import a library and when to implement functionality from scratch. These decisions, whether to build functionality from scratch or buy into an external library, hereafter build-versus-buy, carry direct consequences for software security, licensing compliance, performance, and long-term maintainability. Yet no controlled experimental study has examined what governs build-versus-buy decisions in agentic AI coding tools. Configuration mechanisms, i.e., the means by which developers tailor agentic AI coding tool behavior to a project or workflow, are one of the primary means by which practitioners can influence these decisions. However, it is unclear which configuration mechanisms influence build-versus-buy decisions most effectively. We present a pre-registered protocol to study how configuration mechanisms alter build-versus-buy behavior in two popular agentic AI coding tools: Claude Code and OpenAI Codex. We will execute controlled programming tasks drawn from a benchmark of staged projects, each constructed around identifiable build-versus-buy points, and will manipulate the configuration supplied to each tool, ranging from no configuration, through context files with soft preferences and explicit prohibitions, to Skills (instructions that can be autonomously discovered), MCP-enabled library discovery tools, and permission controls, measuring which libraries the tool selects, whether it discloses newly introduced libraries, and whether those disclosures are complete and accurate. Nine pre-registered hypotheses structure the protocol. The resulting benchmark dataset and analysis pipeline will be released as a reusable artifact for evaluating build-versus-buy behavior in agentic AI coding tools.
翻译:智能体AI编码工具以日益增强的自主性编写代码,在此过程中决定何时导入库以及何时从头实现功能。这些决策——是从头构建功能还是采购外部库(以下称为“自建vs.采购”决策)——对软件安全性、许可合规性、性能和长期可维护性具有直接影响。然而,尚无受控实验研究探讨智能体AI编码工具中“自建vs.采购”决策的驱动因素。配置机制,即开发者针对项目或工作流定制智能体AI编码工具行为的手段,是实践者影响这些决策的主要方式之一。但尚不明确哪些配置机制对“自建vs.采购”决策的影响最为有效。我们提出一项预注册的研究方案,旨在研究配置机制如何改变两款流行的智能体AI编码工具(Claude Code和OpenAI Codex)中的“自建vs.采购”行为。我们将执行基于分阶段项目基准测试的受控编程任务,每个任务围绕可识别的“自建vs.采购”节点构建,并操控提供给每个工具的配置(范围从无配置、带有软偏好和明确禁止的上下文文件,到可自主发现的指令(Skills)、启用MCP的库发现工具以及权限控制),测量工具选择的库、是否披露新引入的库以及这些披露是否完整准确。九项预注册假设构成了该方案的结构。最终的基准数据集和分析管道将作为可复用工件发布,用于评估智能体AI编码工具中的“自建vs.采购”行为。