End-to-end task-success is the dominant way to evaluate LLM agents, but one aggregate number tells you that an agent regressed, not where. We present layer-isolated evaluation: a deployed ordering agent is decomposed into a fixed taxonomy of layers (ontology, intent, routing, decomposition, escalation, safety, memory, and cross-cutting envelope/defense), each exercised by its own assertion slice in a deterministic, no-LLM "pure" mode. The pure suite (238 cases across 23 slices; 225 run in 2.39 s, ~10 ms/case) runs in CI on every change against a locked per-slice baseline. We validate by controlled regression injection, degrading one layer at a time across seven non-safety layers. The effect we did not design in is masking: the aggregate pass-rate barely moves (-1.7 to -5.9 pp for six local regressions), while the matching slice craters (-25 to -91 pp). A layer's slice reacting to its own fault is partly by construction; the measured results are (i) the aggregate masking and (ii) that damage stays off the other slices: the injected layer's slice is the single worst-hit in 5 of 7 cases and top-3 in 7 of 7 (mean rank 1.29 of 19). Localization replicates on a second, structurally different tenant (Starbucks SG): all seven matching slices crater, so it is not a single-catalog artifact. We position it as a concrete, deterministic instantiation of the component-level evaluation EDDOps prescribes but leaves unimplemented, with CheckList as ancestor and as the deterministic mirror image of whole-workflow stochastic mutation testing. Our contributions: (a) a fully decomposed, sub-second, no-LLM per-layer harness for a production agent, (b) a coverage-honesty test-adequacy criterion that refuses to score an unexercised layer, and (c) the regression-injection demonstration that per-slice baseline-locked gates localize regressions an aggregate metric masks.

翻译：端到端任务成功率是评估大语言模型智能体的主流方式，但单一聚合指标仅能告知模型出现回归退化，却无法定位问题根源。本文提出层隔离评估方法：将已部署的订单处理智能体分解为固定分类层次结构（本体层、意图层、路由层、分解层、升级层、安全层、记忆层及横切封装/防御层），每层通过其专属断言切片在确定性、无大语言模型的"纯"模式下进行测试。该纯测试套件包含23个切片的238个测试用例，其中225个用例在2.39秒内完成（约10毫秒/例），每次变更均在持续集成环境中针对锁定的逐层基线进行验证。我们通过受控回归注入实验验证该方法，在七个非安全层中逐层引入退化。未预先设计的"掩蔽效应"涌现：聚合通过率几乎不变（六项局部回归变动区间为-1.7至-5.9个百分点），而对应匹配切片通过率暴跌（-25至-91个百分点）。层切片对其自身故障的响应部分由构造决定；实测结果表明：（i）聚合指标存在掩蔽效应，（ii）故障影响未扩散至其他切片：注入层的切片在7个案例中的5个成为影响最严重的切片，在7个案例中全部位列前三（平均排名1.29/19）。该定位方法在结构不同的第二个租户（新加坡星巴克）上得到复现：全部七个匹配切片均出现暴跌，证明非单目录伪影。我们将该方法定位为EDDOps所规定但未实现组件级评估的具体确定性实例，以CheckList为技术先驱，并作为全工作流随机突变测试的确定性镜像映射。本文贡献包括：（a）为生产级智能体构建完全分解、亚秒级、无大语言模型的逐层测试平台，（b）提出覆盖完整性测试充分性准则，拒绝为未执行层评分，（c）通过回归注入实验证明，基于逐层基线的锁定门控机制能够定位被聚合指标掩盖的回归问题。