Although Large Language Models (LLMs) demonstrate remarkable reasoning capabilities, inherent social biases often cascade throughout the Chain-of-Thought (CoT) process, leading to continuous "Bias Propagation". Existing debiasing methods primarily focus on static constraints or external interventions, failing to identify and interrupt this propagation once triggered. To address this limitation, we introduce Self-Debias, a progressive framework designed to instill intrinsic self-correction capabilities. Specifically, we reformulate the debiasing process as a strategic resource redistribution problem, treating the model's output probability mass as a limited resource to be reallocated from biased heuristics to unbiased reasoning paths. Unlike standard preference optimization which applies broad penalties, Self-Debias employs a fine-grained trajectory-level objective subject to dynamic debiasing constraints. This enables the model to selectively revise biased reasoning suffixes while preserving valid contextual prefixes. Furthermore, we integrate an online self-improvement mechanism utilizing consistency filtering to autonomously synthesize supervision signals. With merely 20k annotated samples, Self-Debias activates efficient self-correction, achieving superior debiasing performance while preserving general reasoning capabilities without continuous external oversight.
翻译:尽管大型语言模型(LLMs)展现出卓越的推理能力,但固有的社会偏见往往会在链式思维(Chain-of-Thought, CoT)过程中级联传播,导致持续的"偏见扩散"。现有去偏方法主要依赖静态约束或外部干预,无法在偏见触发后识别并中断这一传播过程。为解决此局限,我们提出自偏差消除(Self-Debias)——一种渐进式框架,旨在赋予模型内在的自校正能力。具体而言,我们将去偏过程重构为战略性资源重分配问题,将模型输出的概率质量视为有限资源,从有偏启发式路径重新分配至无偏推理路径。与采用宽泛惩罚的标准偏好优化不同,自偏差消除在动态去偏约束下实施细粒度的轨迹级目标,使模型能够选择性修正有偏的推理后缀,同时保留有效的上下文前缀。此外,我们集成基于一致性过滤的在线自我改进机制,自主合成监督信号。仅需2万条标注样本,自偏差消除即可激活高效自校正,在保持通用推理能力的同时实现卓越的去偏性能,且无需持续的外部监督。