Accurately evaluating adversarial robustness is a longstanding challenge. A flawed attack design can inflate robustness estimates, making deployment risk assessment and defense comparison unreliable. Historically, standardized attacks such as AutoAttack have largely resolved this for image classifiers, providing a reliable evaluation baseline for systematic comparison across defenses. However, no equivalent exists for LLM jailbreak evaluation yet, where designing such an attack is considerably more difficult. A reliable attack must, among other things, be black-box compatible, applicable to arbitrary defense pipelines, and efficient, which no existing method jointly satisfies. We introduce Indirect Harm Optimization (IHO), a masked diffusion language model attacker trained via iterative preference optimization against a harmfulness judge, requiring only black-box access to the target. The same method can be used without modification as a strong adaptive attack on individual behaviors, or as an efficient amortized policy that transfers to held-out behaviors and unseen target models without fine-tuning. Even against layered defenses, such as a Circuit Breaker-trained model combined with an auxiliary detector, IHO improves attack success considerably over state-of-the-art approaches, without any defense-specific adaptation. Our results position IHO as a practical step toward the kind of standardized jailbreak evaluation that has improved reliability in the past. Code and models are available on GitHub and Hugging Face.

翻译：准确评估对抗鲁棒性是一项长期挑战。有缺陷的攻击设计会高估鲁棒性指标，导致部署风险评估和防御对比不可靠。历史上，AutoAttack等标准化攻击在很大程度上解决了图像分类器的这一问题，为跨防御系统的系统对比提供了可靠的评估基线。然而，大语言模型越狱评估尚无类似标准，设计此类攻击的难度显著更高。可靠的攻击必须同时满足黑盒兼容性、可应用于任意防御管线、以及高效性等条件，而现有方法均无法同时满足。我们提出间接危害优化（IHO），这是一种通过迭代偏好优化针对危害性评判器训练得到的掩码扩散语言模型攻击器，仅需对目标模型进行黑盒访问。该方法无需修改即可作为针对个体行为的强自适应攻击，或作为高效摊销策略迁移至未见过行为及未见目标模型（无需微调）。即使面对分层防御（如结合辅助检测器的Circuit Breaker训练模型），IHO在不进行任何防御特定适配的情况下，仍能显著提升攻击成功率。我们的结果证明IHO是迈向标准化越狱评估（此类评估曾提升图像分类领域的可靠性）的实用一步。代码和模型已在GitHub和Hugging Face开源。