As Large Language Models (LLMs) are increasingly deployed in safety-critical domains, rigorously evaluating their robustness against adversarial jailbreaks is essential. However, current safety evaluations often overestimate robustness because existing automated attacks are limited by restrictive assumptions. They typically rely on handcrafted priors or require white-box access for gradient propagation. We challenge these constraints by demonstrating that token-level iterative optimization can succeed without gradients or priors. We introduce RAILS (RAndom Iterative Local Search), a framework that operates solely on model logits. RAILS matches the effectiveness of gradient-based methods through two key innovations: a novel auto-regressive loss that enforces exact prefix matching, and a history-based selection strategy that bridges the gap between the proxy optimization objective and the true attack success rate. Crucially, by eliminating gradient dependency, RAILS enables cross-tokenizer ensemble attacks. This allows for the discovery of shared adversarial patterns that generalize across disjoint vocabularies, significantly enhancing transferability to closed-source systems. Empirically, RAILS achieves near 100% success rates on multiple open-source models and high black-box attack transferability to closed-source systems like GPT and Gemini.

翻译：随着大语言模型（LLM）日益部署在安全关键领域，严格评估其对抗越狱攻击的鲁棒性至关重要。然而，当前的安全性评估往往高估了模型的鲁棒性，因为现有的自动化攻击方法受限于严格的假设条件。这些方法通常依赖于手工构建的先验知识，或需要白盒访问权限以进行梯度传播。我们通过证明无需梯度或先验知识的词元级迭代优化同样能够成功，对这些限制提出了挑战。本文提出RAILS（随机迭代局部搜索）框架，该框架仅基于模型的对数概率进行操作。RAILS通过两项关键创新实现了与基于梯度的方法相当的效果：一种强制精确前缀匹配的新型自回归损失函数，以及一种基于历史的选择策略，该策略弥合了代理优化目标与真实攻击成功率之间的差距。至关重要的是，通过消除对梯度的依赖，RAILS实现了跨分词器的集成攻击。这使得系统能够发现可泛化至互不相交词表的共享对抗模式，显著提升了对闭源系统（如GPT和Gemini）的黑盒攻击可迁移性。实验表明，RAILS在多个开源模型上实现了接近100%的成功率，并对闭源系统展现出较高的黑盒攻击可迁移性。