Reliable operation of multi-agent large language model (LLM) systems depends on debugging long execution traces, where the few causally decisive events are buried in unstructured logs of messages, routes, memory writes, and tool calls. The standard tool is counterfactual replay (rewind, edit, and re-run the trajectory to measure each event's effect), but its cost grows linearly with the number of candidate events, making exhaustive replay infeasible at scale. We frame trace debugging as a knowledge-based decision-support problem. Each trace is compiled into a structured event knowledge graph over routing, memory, tool-use, uncertainty, and latent evidence, and a calibrated predictor decides where a scarce replay budget should be spent. We do not propose a new replay oracle; we propose a method to predict its results without paying the replay cost. We formulate zero-replay counterfactual-effect prediction: given a trace under a fixed budget, predict which events the oracle would mark high-effect before any replay is performed. BranchPoint-Latent is a lightweight predictor over observable, structural, uncertainty, and latent features of the knowledge graph. Calibrated against a deterministic replay oracle across 37 trace families, a single learning-to-rank gradient-boosted predictor raises per-trace localization (Branch Recall@5) from 0.73 to 0.93 on held-out families at zero oracle-replay cost. Rather than claiming universal dominance, we characterize when cheap graph centrality suffices and when learned evidence is necessary. The result is an auditable, cost-efficient decision-support system for AI-reliability debugging, positioned explicitly on the cost-accuracy frontier with reproducible artifacts.
翻译:多智能体大语言模型系统的可靠运行依赖于对长执行轨迹的调试,其中少数因果决定性事件通常深埋于消息、路由、内存写入和工具调用的非结构化日志中。标准调试工具是反事实回放(倒回、编辑并重放轨迹以测量每个事件的效果),但其成本随候选事件数量线性增长,使得大规模场景下的穷举回放不可行。本文将轨迹调试形式化为基于知识的决策支持问题。每条轨迹被编译为包含路由、内存、工具使用、不确定性和潜在证据的结构化事件知识图谱,并通过校准预测器决定稀缺回放预算的分配方式。我们不提出新的回放预言机,而是提出一种在不支付回放成本的情况下预测其结果的方案。我们定义了零回放反事实效果预测:在固定预算下,给定一条轨迹,在未执行任何回放前预测预言机会标记为高影响的事件。BranchPoint-Latent是一种轻量级预测器,基于知识图谱的可观测特征、结构特征、不确定性特征和潜在特征。通过37个轨迹族对确定性回放预言机进行校准,单一学习排序梯度提升预测器在零预言机回放成本下,将留出轨迹族的逐轨迹定位(分支召回率@5)从0.73提升至0.93。本文不宣称普适性优势,而是阐明何时简单的图中心性方法足够、何时需要学习型证据。最终成果是一个可审计、成本高效的AI可靠性调试决策支持系统,明确位于成本-精度前沿,并提供可复现的工程制品。