Audio-language models (ALMs) often follow text that conflicts with audio, even when the audio evidence is clear. This raises a basic question: is the audio-supported answer unavailable, or is it represented but overridden by the conflicting text? We examine this question using a same-audio counterfactual that keeps the audio fixed, removes only the conflicting text, and measures the resulting shift in model preference. Across five ALMs and four conflict tasks, 64.1% of conflict samples show a sign flip: the same-audio branch prefers the audio-supported answer, whereas the joint branch prefers the text-supported answer. This pattern suggests that the relevant audio evidence is encoded but loses in arbitration. Activation patching further localizes the reversal to answer-position computation, and patching effects closely track output candidate-score differences (Spearman rho=0.93). Using this diagnostic, we propose Gated Audio Counterfactual Logit Correction (GACL), a training-free decoding rule that interpolates between joint and same-audio scores. Under a strict 5 pp faithfulness-drop budget, GACL improves nAUC by 17.8 points over the best contrastive baseline and transfers without retuning to vision-text arbitration (up to +40.5 pp).

翻译：音频-语言模型（ALMs）常常遵从与音频冲突的文本指令，即便音频证据十分明确。这引出一个基本问题：音频支持的答案是不可获得的，还是已被表征但被冲突文本所覆盖？我们通过使用同音频反事实来研究此问题，该反事实固定音频不变，仅移除冲突文本，并测量模型偏好的相应变化。在五种ALM和四个冲突任务中，64.1%的冲突样本显示出符号翻转：同音频分支倾向于音频支持的答案，而联合分支倾向于文本支持的答案。这一模式表明相关音频证据已被编码，但在仲裁中落败。激活补丁进一步将逆转定位到答案位置的计算，且补丁效应与输出候选分数差异高度相关（Spearman rho=0.93）。基于这一诊断，我们提出门控音频反事实对数几率修正（GACL），一种无需训练的解码规则，用于插值联合分数与同音频分数。在严格5个百分点（pp）的忠实度下降预算下，GACL在nAUC指标上比最优对比基线提升17.8点，并可零调参迁移至视觉-文本仲裁（提升高达40.5 pp）。