Industrial automation increasingly requires flexible control strategies that can adapt to changing tasks and environments. Agents based on Large Language Models (LLMs) offer potential for such adaptive planning and execution but lack standardized benchmarks for systematic comparison. We introduce a benchmark with an executable simulation environment representing the Blocksworld problem providing five complexity categories. By integrating the Model Context Protocol (MCP) as a standardized tool interface, diverse agent architectures can be connected to and evaluated against the benchmark without implementation-specific modifications. A single-agent implementation demonstrates the benchmark's applicability, establishing quantitative metrics for comparison of LLM-based planning and execution approaches.

翻译：工业自动化日益需要能够适应任务与环境变化的灵活控制策略。基于大型语言模型（LLM）的智能体为此类自适应规划与执行提供了潜力，但缺乏用于系统化比较的标准化基准。我们引入了一个包含可执行仿真环境的基准，该环境表示积木世界问题，并提供五个复杂度类别。通过集成模型上下文协议（MCP）作为标准化工具接口，多样化的智能体架构无需针对特定实现进行修改即可连接至该基准并进行评估。一个单智能体实现展示了该基准的适用性，并建立了用于比较基于LLM的规划与执行方法的量化指标。