The adoption of AI-powered Integrated Development Environments (AI IDEs) has introduced "Rules" as a novel software artifact, allowing developers to persistently inject project-specific constraints and architectural guidelines into the context of Large Language Models (LLMs). Despite their role in aligning AI behavior with developer intent, the taxonomy, evolution, and practical impact of these rules remain largely unexplored. To bridge this gap, we conducted a mixed-methods empirical study on AI IDE rules. By mining 83 open-source projects and extracting 7,310 rules, we established a comprehensive taxonomy comprising 5 primary and 25 secondary categories. We then triangulated these artifacts with survey responses from 99 practitioners. Our analysis identified a contrast between developer priorities and actual configurations: while practitioners rate architectural constraints as highly important, rule files in repositories primarily consist of low-level workflow and code formatting constraints. Furthermore, our analysis of 1,540 rule evolution events revealed that rules are updated frequently. Repository data further indicate that rule evolution is primarily driven by constructive context expansions (29.17%) and enrichments (26.59%). In contrast, surveyed developers reported modifying rules primarily to correct AI errors (77.78%), typically by adding new negative constraints rather than editing existing ones. Finally, an artifact compliance assessment of 160 rule evolution events revealed that updating rules significantly improves the adherence of software artifacts, with the average artifact compliance rate increasing by 22.99% (from 49.14% to 72.13%) following an update. Our study provides empirical insights that can help developers optimize prompting strategies and guide tool builders in designing automated conflict-detection and context-management mechanisms for AI IDEs.
翻译:人工智能增强型集成开发环境(AI IDE)的采用引入了“规则”这一新型软件制品,允许开发者持续将特定项目的约束条件和架构指南注入大语言模型(LLM)的上下文中。尽管这些规则在协调AI行为与开发者意图方面发挥关键作用,但其分类体系、演化规律及实际影响仍缺乏充分探索。为填补这一空白,我们开展了一项关于AI IDE规则的混合方法实证研究。通过挖掘83个开源项目并提取7,310条规则,我们构建了一套包含5个主类别和25个子类别的综合分类体系。随后,我们将这些发现与99名从业者的调查反馈进行三角验证。分析揭示出开发者优先级与实际配置之间的差异:尽管从业者认为架构约束至关重要,但仓库中的规则文件主要包含低层级工作流和代码格式约束。此外,对1,540次规则演化事件的分析表明,规则更新频率较高。仓库数据进一步显示,规则演化主要由建设性上下文扩展(29.17%)和丰富化(26.59%)驱动。相比之下,受访开发者报告修改规则的主要动机是纠正AI错误(77.78%),通常通过新增负面约束而非编辑现有约束实现。最后,对160次规则演化事件的制品合规性评估表明,更新规则能显著提升软件制品的遵循程度——更新后制品的平均合规率从49.14%提升至72.13%,增幅达22.99%。本研究提供的实证见解可帮助开发者优化提示策略,并指导工具构建者为AI IDE设计自动化冲突检测与上下文管理机制。