LLM-powered tool-calling agents fulfill user requests by interacting with environments, querying data, and invoking tools in a multi-turn process. Yet, most existing benchmarks evaluate these systems under static environment interfaces, with fixed schemas and toolsets, making it difficult to assess how agents behave as environments evolves -- when capabilities are added, reorganized, or deprecated across successive environment versions. In this paper, we study structured environment evolution as a benchmark-construction problem for tool-calling agents. We propose ProEvolve, a graph-based framework that makes environment evolution programmable. At its core, a typed relational graph provides a unified, explicit representation of the environment - data, tools, and schema. Under this formalism, adding, removing, or modifying capabilities are expressed as graph transformations that coherently propagate updates across tools, schemas, and data access. Building on this, ProEvolve supports (1) automatic generation of evolved executable environments through explicit graph transformations, and (2) graph-grounded construction of task sandboxes via subgraph sampling and instantiation. We validate ProEvolve in two tool-calling domains, e-commerce and airline booking, in terms of quality, implementation validity, and failure modes. Finally, we use the generated benchmark as a downstream diagnostic to study how representative agents behave under structured environment evolution.

翻译：基于大语言模型的工具调用智能体通过与环境交互、查询数据和多轮调用函数来满足用户需求。然而，现有基准测试大多在静态环境接口下评估这些系统——具有固定模式、固定工具集的结构——难以评估当环境发生演进（例如在连续版本环境中新增、重组或废弃功能）时智能体的行为表现。本文从基准测试构建角度研究结构化环境演化问题，提出基于图的ProEvolve框架实现环境演化的可编程性。该框架以类型化关系图为核心，为环境（数据、工具和模式）提供统一显式表征。在此形式化体系下，能力的增删改操作被表达为能在工具、模式和数据访问间实现连贯更新的图变换操作。基于此，ProEvolve支持：（1）通过显式图变换自动生成演化后的可执行环境；（2）通过子图采样与实例化构建基于图的任务沙箱。我们在电子商务和机票预订两个工具调用领域验证了ProEvolve的质量、实现有效性及失败模式。最后，将生成的基准测试作为下游诊断工具，研究代表性智能体在结构化环境演化中的行为表现。