Agents aspire to eliminate the need for task-specific prompt crafting through autonomous reason-act-observe loops. Still, they are commonly instructed to follow a task-specific plan for guidance, e.g., to resolve software issues following phases for navigation, reproduction, patch, and validation. Unfortunately, it is unknown to what extent agents actually follow such instructed plans. Without such an analysis, determining the extent agents comply with a given plan, it is impossible to assess whether a solution was reached through correct strategic reasoning or through other means, e.g., data contamination or overfitting to a benchmark. This paper presents the first extensive, systematic analysis of plan compliance in programming agents, examining 16,991 trajectories from SWE-agent across four LLMs on SWE-bench Verified and SWE-bench Pro under eight plan variations. Without an explicit plan, agents fall back on workflows internalized during training, which are often incomplete, overfit, or inconsistently applied. Providing the standard plan improves issue resolution, and we observe that periodic plan reminders can mitigate plan violations and improve task success. A subpar plan hurts performance even more than no plan at all. Surprisingly, augmenting a plan with additional task-relevant phases in the early stage can degrade performance, particularly when these phases do not align with the model's internal problem-solving strategy. These findings highlight a research gap: fine-tuning paradigms that teach models to follow instructed plans, rather than encoding task-specific plans in them. This requires teaching models to reason and act adaptively, rather than memorizing workflows.

翻译：智能体通过自主"推理-行动-观察"循环，致力于消除对特定任务提示设计的依赖。然而，通常仍需遵循任务导向的计划指导，例如，解决软件问题时需按导航、复现、补丁、验证等阶段执行。但当前尚不明确智能体在多大程度上实际遵循此类指令性计划。若缺乏对计划遵循度的分析，就无法确定解决方案是通过正确的策略推理达成，还是通过其他手段（如数据污染或对基准的过拟合）实现。本文首次对编程智能体的计划遵循度进行了系统性大规模分析，基于SWE-bench Verified与SWE-bench Pro基准测试，在SWE-agent框架下考察了四种大语言模型在八种计划变体中的16,991条执行轨迹。研究发现：缺乏显式计划时，智能体会退回到训练中内化的流程，这些流程往往不完整、过拟合或应用不一致；提供标准计划可提升问题解决率，而周期性计划提醒能减少违规行为并提高任务成功率；劣质计划对性能的损害甚至超过无计划状态。令人意外的是，在计划早期阶段增加任务相关环节可能降低性能，尤其是当这些环节与模型内置的问题解决策略存在冲突时。这些发现揭示了当前研究空白：需要开发微调范式教会模型遵循指令性计划，而非将特定任务计划编码于模型中——这要求模型学会适应性推理与行动，而非机械记忆工作流程。