Large language models (LLMs) frequently produce false refusals, declining benign requests that contain terms resembling unsafe queries. We address this challenge by introducing two comprehensive benchmarks: the Exaggerated Safety Benchmark (XSB) for single-turn prompts, annotated with "Focus" keywords that identify refusal-inducing triggers, and the Multi-turn Scenario-based Exaggerated Safety Benchmark (MS-XSB), which systematically evaluates refusal calibration in realistic, context-rich dialog settings. Our benchmarks reveal that exaggerated refusals persist across diverse recent LLMs and are especially pronounced in complex, multi-turn scenarios. To mitigate these failures, we leverage post-hoc explanation methods to identify refusal triggers and deploy three lightweight, model-agnostic approaches, ignore-word instructions, prompt rephrasing, and attention steering, at inference time, all without retraining or parameter access. Experiments on four instruction-tuned Llama models demonstrate that these strategies substantially improve compliance on safe prompts while maintaining robust safety protections. Our findings establish a reproducible framework for diagnosing and mitigating exaggerated refusals, highlighting practical pathways to safer and more helpful LLM deployments.

翻译：大语言模型（LLMs）常产生虚假拒绝，即对包含类似不安全查询术语的良性请求予以拒绝。为解决此问题，我们引入两个综合性基准：针对单轮提示的夸大安全基准（XSB），其标注了识别拒绝诱导触发器的“焦点”关键词；以及多轮场景化夸大安全基准（MS-XSB），用于系统评估现实、上下文丰富的对话设置中的拒绝校准。我们的基准测试表明，夸大拒绝现象在多种近期LLMs中持续存在，且在复杂的多轮场景中尤为显著。为缓解此类故障，我们利用事后解释方法识别拒绝触发器，并在推理时部署三种轻量级、模型无关的方法——忽略词指令、提示重述和注意力引导，所有方法均无需重新训练或参数访问。在四个指令调优的Llama模型上的实验表明，这些策略能显著提升对安全提示的遵从性，同时保持稳健的安全防护。我们的研究建立了一个可复现的诊断与缓解夸大拒绝的框架，为更安全、更有助益的LLM部署指明了实用路径。