Large Language Model safety alignment predominantly operates on a binary assumption that requests are either safe or unsafe. This classification proves insufficient when models encounter ethical dilemmas, where the capacity to reason through moral trade-offs creates a distinct attack surface. We formalize this vulnerability through TRIAL, a multi-turn red-teaming methodology that embeds harmful requests within ethical framings. TRIAL achieves high attack success rates across most tested models by systematically exploiting the model's ethical reasoning capabilities to frame harmful actions as morally necessary compromises. Building on these insights, we introduce ERR (Ethical Reasoning Robustness), a defense framework that distinguishes between instrumental responses that enable harmful outcomes and explanatory responses that analyze ethical frameworks without endorsing harmful acts. ERR employs a Layer-Stratified Harm-Gated LoRA architecture, achieving robust defense against reasoning-based attacks while preserving model utility.

翻译：大型语言模型的安全对齐主要基于请求非安全即安全的二元假设。当模型面临伦理困境时，这种分类方式显得不足，因为权衡道德得失的推理能力会形成独特的攻击面。我们通过TRIAL方法对这种脆弱性进行形式化建模——这是一种将有害请求嵌入伦理框架的多轮红队测试方法。TRIAL通过系统性地利用模型的伦理推理能力，将有害行为构建为道德上必要的妥协，从而在大多数测试模型中实现了较高的攻击成功率。基于这些发现，我们提出ERR（伦理推理鲁棒性）防御框架，该框架能够区分导致有害结果的工具性回应与仅分析伦理框架但不支持有害行为的解释性回应。ERR采用分层式危害门控LoRA架构，在保持模型实用性的同时，实现了对基于推理攻击的鲁棒防御。