AI agents often fail in ways that are difficult to localize because executions are probabilistic, long-horizon, multi-agent, and mediated by noisy tool outputs. We address this gap by manually annotating failed agent runs and release a novel benchmark of 115 failed trajectories spanning structured API workflows, incident management, and open-ended web/file tasks. Each trajectory is annotated with a critical failure step and a category from a grounded-theory derived, cross-domain failure taxonomy. To mitigate the human cost of failure attribution, we present AGENTRX, an automated domain-agnostic diagnostic framework that pinpoints the critical failure step in a failed agent trajectory. It synthesizes constraints, evaluates them step-by-step, and produces an auditable validation log of constraint violations with associated evidence; an LLM-based judge uses this log to localize the critical step and category. Our framework improves step localization and failure attribution over existing baselines across three domains.

翻译：AI智能体常因执行过程的概率性、长时程、多智能体特性及噪声工具输出的中介作用，其故障难以准确定位。本研究通过人工标注故障智能体运行轨迹，构建了包含115个故障轨迹的新型基准数据集，涵盖结构化API工作流、事件管理与开放式网页/文件任务三大领域。每个轨迹均标注了关键故障步骤，并依据扎根理论推导的跨领域故障分类体系进行归类。为降低故障归因的人工成本，我们提出AGENTRX——一种领域无关的自动化诊断框架，可精准定位故障智能体轨迹中的关键失效步骤。该框架通过综合约束条件、逐步骤评估约束满足度，生成包含约束违反证据的可审计验证日志；基于LLM的判定器利用该日志定位关键步骤及故障类别。实验表明，本框架在三个不同领域中，其步骤定位与故障归因能力均优于现有基线方法。