Large language models are meticulously aligned to be both helpful and harmless. However, recent research points to a potential overkill which means models may refuse to answer benign queries. In this paper, we investigate the factors for overkill by exploring how models handle and determine the safety of queries. Our findings reveal the presence of shortcuts within models, leading to an over-attention of harmful words like 'kill' and prompts emphasizing safety will exacerbate overkill. Based on these insights, we introduce Self-Contrastive Decoding (Self-CD), a training-free and model-agnostic strategy, to alleviate this phenomenon. We first extract such over-attention by amplifying the difference in the model's output distributions when responding to system prompts that either include or omit an emphasis on safety. Then we determine the final next-token predictions by downplaying the over-attention from the model via contrastive decoding. Empirical results indicate that our method has achieved an average reduction of the refusal rate by 20\% while having almost no impact on safety.

翻译：大型语言模型经过精细对齐，旨在兼顾有用性与无害性。然而，近期研究指出存在过度防御问题，即模型可能拒绝回答良性查询。本文通过探究模型如何处理及判断查询安全性，揭示了过度防御的影响因素。研究发现模型内部存在捷径机制，导致对"杀戮"等危害性词汇产生过度关注，且强调安全的提示会加剧过度防御现象。基于此，我们提出无需训练且模型无关的自对比解码策略以缓解该问题。首先，通过放大模型在包含/省略安全强调的系统提示下输出分布的差异，提取此类过度关注特征；继而采用对比解码方式弱化模型中的过度关注，确定最终下一个词预测。实验结果表明，该方法在平均降低20%拒绝率的同时，几乎不影响模型安全性。