Large language models (LLMs) increasingly serve as automated judges, yet they remain susceptible to cognitive biases -- often altering their reasoning when faced with spurious prompt-level cues such as consensus claims or authority appeals. Existing mitigations via prompting or supervised fine-tuning fail to generalize, as they modify surface behavior without changing the optimization objective that makes bias cues predictive. To address this gap, we propose Epistemic Independence Training (EIT), a reinforcement learning framework grounded in a key principle: to learn independence, bias cues must be made non-predictive of reward. EIT operationalizes this through a balanced conflict strategy where bias signals are equally likely to support correct and incorrect answers, combined with a reward design that penalizes bias-following without rewarding bias agreement. Experiments on Qwen3-4B demonstrate that EIT improves both accuracy and robustness under adversarial biases, while preserving performance when bias aligns with truth. Notably, models trained only on bandwagon bias generalize to unseen bias types such as authority and distraction, indicating that EIT induces transferable epistemic independence rather than bias-specific heuristics. Code and data are available at https://anonymous.4open.science/r/bias-mitigation-with-rl-BC47.

翻译：大型语言模型（LLM）日益充当自动化评判器，但它们仍然容易受到认知偏见的影响——在面对虚假的提示层面线索（如共识主张或权威诉求）时，常常改变其推理过程。现有的通过提示或监督微调的缓解方法未能实现泛化，因为它们仅改变了表面行为，而未改变使偏见线索具有预测性的优化目标。为弥补这一不足，我们提出了认知独立性训练（EIT），这是一个基于关键原则的强化学习框架：要学习独立性，必须使偏见线索对奖励不具有预测性。EIT通过一种平衡冲突策略来实现这一点，其中偏见信号同等可能支持正确和错误答案，并结合一种奖励设计，该设计惩罚遵循偏见的行为而不奖励与偏见一致的行为。在Qwen3-4B上的实验表明，EIT在对抗性偏见下提高了准确性和鲁棒性，同时在偏见与真相一致时保持了性能。值得注意的是，仅在从众偏见上训练的模型能够泛化到未见过的偏见类型（如权威和干扰），这表明EIT诱导了可迁移的认知独立性，而非特定于偏见的启发式方法。代码和数据可在 https://anonymous.4open.science/r/bias-mitigation-with-rl-BC47 获取。