Temperature-zero BF16 LLM inference is often treated as reproducible, yet the same request can emit different tokens when decoded alone or inside a larger batch. Existing fixes use batch-invariant operators or LLM-42's per-token verification, incurring cost even when most steps are stable. We ask whether verification can be applied exclusively to flipped tokens. Across five models, batch-induced token flips are sparse on the flip-rate benchmarks: on MATH500, Llama-3.1-8B flips on $0.48\%$ of synchronous decode steps, and all tested models stay within the 0.3-1.3% range on MATH500, GSM8K, and HumanEval. K/V perturbations remain flat before flips, while low top-1/top-2 logit margins expose much of the flip risk. MarginGate turns these observations into a verifier policy: it keeps BF16 decoding on high-margin steps, verifies only low-margin steps, and repairs confirmed mismatches by replacing the current K/V column. We evaluate on four datasets, calibrating on MATH500 and transferring to GSM8K, SharedGPT, and HumanEval. MarginGate restores 100% sequence-level deterministic decoding on Llama-3.1-8B and Qwen2.5-14B with 18.56%/15.05% verifier trigger rates, reducing LLM-42's latency increment by 2.23x/1.99x relative to always-on verification. On DSR1-Distill-Qwen-7B, the same policy reaches determinism in a harder regime at 49.50% triggers.

翻译：零温度BF16大语言模型推理通常被认为是可重现的，但同一请求在单独解码与批量解码时可能输出不同词元。现有修正方案采用批量不变算子或LLM-42的逐词元验证机制，即使多数解码步骤稳定仍产生开销。我们提出疑问：验证能否仅应用于翻转词元？在五个模型上，批量引发的词元翻转在翻转率基准测试中呈现稀疏性：在MATH500数据集上，Llama-3.1-8B在同步解码步骤中的翻转率为0.48%，所有测试模型在MATH500、GSM8K和HumanEval上的翻转率均维持在0.3-1.3%范围内。词元翻转前K/V扰动保持平稳，而低top-1/top-2 logit边际暴露了大部分翻转风险。MarginGate将这些观察转化为验证策略：对高边际步骤保持BF16解码，仅验证低边际步骤，并通过替换当前K/V列修复已确认的不匹配。我们在四个数据集上进行评估，以MATH500校准并迁移至GSM8K、SharedGPT和HumanEval。MarginGate在验证触发率为18.56%/15.05%的条件下，使Llama-3.1-8B和Qwen2.5-14B实现100%的序列级确定性解码，相比始终开启的验证方案，将LLM-42的延迟增幅降低了2.23倍/1.99倍。在DSR1-Distill-Qwen-7B上，同一策略在49.50%触发率下实现了更严苛场景中的确定性解码。