AI agents are rapidly advancing from passive language models to autonomous systems executing complex, multi-step tasks. Yet their overconfidence in failure remains a fundamental barrier to deployment in high-stakes settings. Existing calibration methods, built for static single-turn outputs, cannot address the unique challenges of agentic systems, such as compounding errors along trajectories, uncertainty from external tools, and opaque failure modes. To address these challenges, we introduce, for the first time, the problem of Agentic Confidence Calibration and propose Holistic Trajectory Calibration (HTC), a novel diagnostic framework that extracts rich process-level features ranging from macro dynamics to micro stability across an agent's entire trajectory. Powered by a simple, interpretable model, HTC consistently surpasses strong baselines in both calibration and discrimination, across eight benchmarks, multiple LLMs, and diverse agent frameworks. Beyond performance, HTC delivers three essential advances: it provides interpretability by revealing the signals behind failure, enables transferability by applying across domains without retraining, and achieves generalization through a General Agent Calibrator (GAC) that achieves the best calibration (lowest ECE) on the out-of-domain GAIA benchmark. Together, these contributions establish a new process-centric paradigm for confidence calibration, providing a framework for diagnosing and enhancing the reliability of AI agents.

翻译：人工智能智能体正迅速从被动的语言模型演变为执行复杂多步骤任务的自主系统。然而，它们在失败情况下的过度自信仍然是高风险场景部署的根本障碍。现有的校准方法专为静态单轮输出设计，无法应对智能体系统的独特挑战，例如轨迹中的误差累积、外部工具的不确定性以及不透明的故障模式。为应对这些挑战，我们首次提出了智能体置信度校准问题，并提出了整体轨迹校准（HTC）——一种新颖的诊断框架，该框架能够从智能体整个轨迹中提取丰富的流程级特征，涵盖从宏观动态到微观稳定性的多个维度。借助一个简单、可解释的模型，HTC在八个基准测试、多种大型语言模型及不同智能体框架中，在校准和判别能力方面均持续超越强基线方法。除性能优势外，HTC还实现了三项重要进展：通过揭示故障背后的信号提供可解释性；无需重新训练即可跨领域应用，实现可迁移性；通过通用智能体校准器（GAC）实现泛化能力，该校准器在跨域GAIA基准测试中取得了最佳校准效果（最低ECE）。这些贡献共同建立了一种以流程为中心的新型置信度校准范式，为诊断和提升人工智能智能体的可靠性提供了框架。