We study a protocol-level test for weak-label benchmarks: whether benchmark outputs change when the provided evidence is intervened on. Metadata-only shortcut checks answer a different question, namely whether outputs are predictable from metadata priors. We therefore combine a metadata statistic, the Metadata Prior Dominance Score (MPDS), with an evidence-intervention statistic, ΔEvi, measuring sensitivity to evidence identity under cross-item shuffling. Synthetic HotpotQA gives a constructed counterexample to metadata-only screening: MPDS is only moderate (0.643), yet ΔEvi is zero. Stronger-reader reruns show why calibration belongs in the test procedure: SNLI shows a calibration reversal, reconstructed HotpotQA occupies a question-dominant warning region, and FEVER is a strongly evidence-sensitive positive control across four transformers. The practical lesson is simple: benchmark audits should report metadata-only screening, evidence intervention, and reader-strength calibration together.

翻译：我们研究了弱标签基准的协议级别测试：当提供的证据受到干预时，基准输出是否发生变化。仅基于元数据的快捷检查回答的是不同问题，即输出是否可从元数据先验中预测。因此，我们将元数据统计量——元数据先验主导得分(MPDS)——与证据干预统计量ΔEvi相结合，后者通过跨项目随机打乱测量对证据身份的敏感性。合成HotpotQA为纯粹元数据筛查提供了一个构造反例：MPDS仅为中等水平(0.643)，但ΔEvi为零。更强阅读器重跑实验表明，校准为何应纳入测试流程：SNLI呈现校准反转现象，重构HotpotQA占据问题主导预警区域，而FEVER在四种Transformer架构中均表现为强证据敏感性阳性对照。实际启示简单明了：基准审计应同时报告元数据筛查、证据干预和阅读器强度校准三方面结果。