Large language models (LLMs) are increasingly deployed in settings that require nuanced ethical reasoning, yet existing bias evaluations treat model outputs as simply "biased" or "unbiased." This binary framing misses the gradual, context-sensitive way bias actually emerges. We address this gap in two stages: behavioral profiling and mechanistic validation. In the behavioral stage, we introduce the Moral Sensitivity Index (MSI), a metric that quantifies the probability of biased output across a graduated, seven-tier stress test ranging from abstract numerical problems to scenarios rooted in historical and socioeconomic injustice. Evaluating four leading models (Claude 3.5, Qwen 3.5, Llama 3, and Gemini 1.5), we identify distinct behavioral signatures shaped by alignment design: for instance, Gemini 1.5 reaches 72.7% MSI by Tier 5 under socioeconomic framing, while Claude exhibits sharp suppression consistent with identity-based safety training. We then verify these behavioral patterns mechanistically. We select criminal-bias scenarios, which produced the highest MSI scores across models, as probes and apply logit lens, attention analysis, activation patching, and semantic probing to a controlled set of six models spanning three capability tiers: small language models (SLMs), instruction-tuned base models, and reasoning-distilled variants. Circuit-level analysis reveals a U-curve of bias: SLMs exhibit strong criminal bias; scaling to instruction-tuned models eliminates it; reasoning distillation reintroduces bias to SLM-like levels despite identical parameter counts, suggesting distillation compresses reasoning traces in ways that reactivate shallow statistical associations. Critically, the socially loaded cues that drive high MSI scores activate the same bias-driving circuits identified mechanistically, providing cross-stage validation.
翻译:大语言模型(LLMs)日益被部署在需要精细伦理推理的场景中,然而现有偏见评估仅将模型输出简单划分为"有偏见"或"无偏见"。这种二元框架忽视了偏见实际展现出的渐进式情境敏感性。我们通过两个阶段的研究填补这一空白:行为特征分析与机制验证。在行为分析阶段,我们提出道德敏感性指数(MSI),该指标通过从抽象数值问题到基于历史与社会经济不公场景的七级渐进式压力测试,量化模型产生偏见输出的概率。评估四个领先模型(Claude 3.5、Qwen 3.5、Llama 3和Gemini 1.5)后,我们识别出由对齐设计塑造的差异化行为特征:例如,Gemini 1.5在社会经济框架下的第五级测试中MSI达到72.7%,而Claude则展现出与基于身份的安全训练一致的强抑制效应。随后我们从机制层面验证这些行为模式。选取跨模型MSI评分最高的刑事偏见场景作为探针,对跨越三个能力层级(小语言模型(SLMs)、指令微调基础模型与推理蒸馏变体)的六个受控模型组,应用logit透镜、注意力分析、激活补丁和语义探针技术。电路级分析揭示了偏见的U型曲线:SLMs表现出强烈刑事偏见;扩大至指令微调模型后偏见消失;而推理蒸馏在参数规模相同时将偏见重新引至接近SLM水平,表明蒸馏以某种方式压缩推理轨迹,该过程重新激活了浅层统计关联。关键的是,驱动高MSI评分的社会性负载线索激活了机制层面已识别的相同偏见驱动电路,这为跨阶段验证提供了实证支持。