Foundation model agents increasingly operate in multi-agent deployments where a coordinator must decide which agent's response to trust. The standard approach weights agents by their self-reported confidence, but recent evidence shows that foundation model confidence is systematically miscalibrated and, on hard tasks, inversely correlated with accuracy. Design-time calibration methods (temperature scaling, Platt scaling, histogram binning) cannot address this problem because they fit a fixed correction to held-out data and degrade under distribution shift. We present MARGIN (Multi-Agent Runtime Grading via Incremental Normalisation), an online calibration method that learns per-agent, per-confidence-band calibration factors from the task stream itself, requiring no model access, no held-out data, and no retraining. MARGIN uses symmetric exponentially weighted moving averages with Bayesian shrinkage blending, and has three hyperparameters with robust defaults. Across 18 foundation models, 8 benchmarks, and over 44,000 observations, MARGIN achieves 3-6x lower calibration error than the best design-time baseline under distribution shift. In multi-agent selection, raw verbalized confidence fails to beat random at pairwise resolution (43-50%) on hard benchmarks. MARGIN corrects this completely, raising pairwise resolution to 70-89% and closing 37-78% of the Raw-to-Oracle pass@1 gap across the five code-generation benchmarks without any oracle knowledge of which model is strongest. Six formal propositions characterize convergence, tracking speed, and the optimality of symmetric updates for non-strategic agents, with all predictions illustrated empirically.

翻译：基础模型智能体越来越多地部署在多智能体环境中，其中协调者必须决定信任哪个智能体的响应。标准方法根据智能体自我报告的置信度进行加权，但近期证据表明，基础模型的置信度存在系统性校准偏差，且在困难任务中与准确率呈反相关。设计时校准方法（温度缩放、Platt缩放、直方图分箱）无法解决此问题，因为它们对保留数据拟合固定的修正项，并在分布偏移下性能退化。我们提出MARGIN（通过增量归一化的多智能体运行时分级），一种在线校准方法，从任务流本身学习每个智能体、每个置信度区间的校准因子，无需模型访问、无需保留数据、无需重新训练。MARGIN采用带贝叶斯收缩混合的对称指数加权移动平均，并包含三个具有稳健默认值的超参数。在18个基础模型、8个基准测试和超过44,000个观测数据上，MARGIN在分布偏移下实现比最优设计时基线低3-6倍的校准误差。在多智能体选择中，原始言语化置信度在困难基准测试的成对分辨率（43-50%）上无法击败随机选择。MARGIN完全纠正了这一问题，将成对分辨率提升至70-89%，并在五个代码生成基准测试中缩小了37-78%的原始到Oracle pass@1差距，无需任何关于哪个模型最强的先验知识。六个形式化命题刻画了非策略型智能体的收敛性、跟踪速度以及对称更新的最优性，所有预测均通过经验验证进行说明。