Large language models deployed for MAPDL finite-element simulation face practical reliability challenges: without structured execution control, tool encapsulation, and fault recovery, outputs may be inconsistent and task failures are common. The Agent Harness paradigm addresses this by inserting domain-specific orchestration middleware that manages tool lifecycles, workflow state, and recovery escalation. This paper presents the architecture of CAX-Agent, a lightweight agent harness purpose-built for MAPDL automation, and empirically evaluates one of its core components -- the recovery policy.CAX-Agent organizes execution into three layers -- LLM service, agent harness, and solver backend -- with a recovery ladder that escalates from deterministic rule patching through model-driven regeneration to context enrichment and human intervention. We evaluate three recovery strategies (no_recovery, rule_only, and model_only) on 50 standard structural benchmarks with three repeated runs per strategy (450 case-runs total). Two independent human raters score task completion under blind conditions; inter-rater agreement is strong (quadratic weighted Cohen's kappa = 0.84, 96 percent of score pairs within one point). Model_only achieves the best completion rate (0.9267), task score (3.59/4), total score (9.16/10), and zero-intervention rate (0.84), outperforming rule_only (0.7733, 3.17/4, 7.03/10, 0.00) and no_recovery (0.6933, 2.74/4, 5.60/10, 0.00) with large effect sizes (Cliff's delta = 0.81-0.87). The benchmark uses deliberately simple geometries to isolate recovery-policy effects; we discuss the scope of these findings and directions for broader validation.
翻译:面向MAPDL有限元仿真部署的大语言模型面临实际可靠性挑战:缺乏结构化执行控制、工具封装与故障恢复机制会导致输出不一致并频繁出现任务失败。Agent工具箱范式通过插入领域特定的编排中间件来管理工具生命周期、工作流状态与恢复升级策略,从而解决该问题。本文提出专为MAPDL自动化设计的轻量级Agent工具箱CAX-Agent的架构,并实证评估其核心组件之一——恢复策略。CAX-Agent将执行过程组织为三层架构(LLM服务层、Agent工具箱层与求解器后端层),并采用阶梯式恢复机制:从确定性规则修补,经模型驱动式重新生成,逐步升级至上下文增强乃至人工干预。我们在50个标准结构基准测试中评估三种恢复策略(无恢复、仅规则、仅模型),每个策略重复运行三次(共450个案例运行)。两名独立人类评分员在盲测条件下对任务完成度进行评分,评分者间一致性较强(二次加权Cohen's kappa=0.84,96%的评分对相差不超过1分)。仅模型策略取得最优完成率(0.9267)、任务评分(3.59/4)、总分(9.16/10)及零干预率(0.84),显著优于仅规则策略(0.7733、3.17/4、7.03/10、0.00)和无恢复策略(0.6933、2.74/4、5.60/10、0.00),效应量较大(Cliff's delta=0.81-0.87)。基准测试采用刻意简化的几何形状以隔离恢复策略效应,本文讨论该发现的适用范围及更广泛验证的方向。