Large Language Model (LLM) code agents increasingly resolve repository-level issues by iteratively editing code, invoking tools, and validating candidate patches. In these workflows, agents often write tests on the fly, a paradigm adopted by many high-ranking agents on the SWE-bench leaderboard. However, we observe that GPT-5.2, which writes almost no new tests, can even achieve performance comparable to top-ranking agents. This raises the critical question: whether such tests meaningfully improve issue resolution or merely mimic human testing practices while consuming a substantial interaction budget. To reveal the impact of agent-written tests, we present an empirical study that analyzes agent trajectories across six state-of-the-art LLMs on SWE-bench Verified. Our results show that while test writing is commonly adopted, but resolved and unresolved tasks within the same model exhibit similar test-writing frequencies Furthermore, these tests typically serve as observational feedback channels, where agents prefer value-revealing print statements significantly more than formal assertion-based checks. Based on these insights, we perform a controlled experiment by revising the prompts of four agents to either increase or reduce test writing. The results suggest that changes in the volume of agent-written tests do not significantly change final outcomes. Taken together, our study reveals that current test-writing practices may provide marginal utility in autonomous software engineering tasks.

翻译：大型语言模型（LLM）代码智能体正日益通过迭代编辑代码、调用工具和验证候选补丁来解决仓库级别的问题。在这些工作流程中，智能体通常会即时编写测试，这一范式已被SWE-bench排行榜上许多排名靠前的智能体所采用。然而，我们观察到几乎不编写新测试的GPT-5.2，其性能甚至可与顶级智能体相媲美。这引发了一个关键问题：此类测试是否能有效提升问题解决能力，抑或仅仅是在消耗大量交互预算的同时模仿人类测试实践。为揭示智能体编写测试的影响，我们开展了一项实证研究，分析了六种最先进LLM在SWE-bench Verified基准上的智能体执行轨迹。结果表明，尽管编写测试被普遍采用，但同一模型内已解决和未解决的任务却表现出相似的测试编写频率。此外，这些测试通常仅作为观察性反馈渠道，智能体明显更倾向于使用揭示变量值的打印语句，而非基于断言的正式检查。基于这些发现，我们通过修改四个智能体的提示词来增加或减少测试编写，进行了一项对照实验。结果显示，智能体编写测试数量的变化并未显著改变最终结果。综上所述，我们的研究表明，在当前的自主软件工程任务中，现有的测试编写实践可能仅提供有限的效用。