Large Language Model (LLM) agents are increasingly applied to engineering design tasks, yet existing evaluation frameworks do not adequately address multi-agent systems that combine simulation, retrieval, and manufacturing preparation. We introduce a benchmark suite with three evaluation dimensions: (1) a workflow benchmark with seven prompt styles targeting distinct cognitive demands-including direct tool use, semantic disambiguation, conditional branching, and working-memory tasks; (2) a Retrieval-Augmented Generation (RAG) benchmark with gated scoring isolating retrieval contributions to parameter selection; and (3) an High Performance Computing (HPC) benchmark evaluating end-to-end ML training orchestration on a SLURM cluster. Alongside the benchmark we present EngiAI, a Multi-Agent System (MAS) reference implementation built on LangGraph that operationalizes the benchmark by coordinating seven specialized agents through a supervisor architecture, unifying topology optimization, document retrieval, HPC job orchestration, and 3D printer control. Across four LLM backends and two EngiBench problems, proprietary models achieve 96-97% average task completion on Beams2D, while open-source 4B-parameter models reach 55-78%, with clear generational improvement. Conditional branching proves most challenging, with task completion dropping to 20-53% for the conditional style on Photonics2D. RAG gating confirms near-perfect retrieval-augmented scores ($\approx 1.0$) versus near-zero without retrieval, validating the evaluation design. On HPC orchestration, one model completes all pipeline steps in 100% of runs while another drops to 50%, revealing that multi-step instruction following degrades over long-running workflows.

翻译：大语言模型（LLM）智能体正越来越多地被应用于工程设计任务，但现有评估框架未能充分涵盖结合仿真、检索和制造准备的多智能体系统。我们提出了一套包含三个评估维度的基准测试套件：（1）一个工作流基准测试，包含七种针对不同认知需求的提示风格——涵盖直接工具使用、语义消歧、条件分支和工作记忆任务；（2）一个检索增强生成（RAG）基准测试，采用门控评分机制，隔离检索对参数选取的贡献；（3）一个高性能计算（HPC）基准测试，用于评估在SLURM集群上进行的端到端机器学习训练编排。除基准测试外，我们还提出了EngiAI——一个基于LangGraph构建的多智能体系统（MAS）参考实现，通过监督者架构协调七个专用智能体，将拓扑优化、文档检索、HPC作业编排和3D打印机控制统一起来，从而将基准测试付诸实践。在四个LLM后端和两个EngiBench问题上，专有模型在Beams2D上的平均任务完成率达到96-97%，而开源4B参数模型达到55-78%，并呈现出明显的代际改进。条件分支最具挑战性，在Photonics2D上条件风格的任务完成率降至20-53%。RAG门控验证了近乎完美的检索增强得分（$\approx 1.0$），而无检索时得分近乎为零，从而证实了评估设计的有效性。在HPC编排方面，一个模型在所有运行中均100%完成了所有流水线步骤，而另一个模型则降至50%，这表明多步骤指令遵循在长工作流中会退化。