Large Language Models remain vulnerable to adversarial prompts that elicit toxic content even after safety alignment. We present ToxSearch, a black-box evolutionary framework that tests model safety by evolving prompts in a synchronous steady-state loop. The system employs a diverse set of operators, including lexical substitutions, negation, back-translation, paraphrasing, and two semantic crossover operators, while a moderation oracle provides fitness guidance. Operator-level analysis shows heterogeneous behavior: lexical substitutions offer the best yield-variance trade-off, semantic-similarity crossover acts as a precise low-throughput inserter, and global rewrites exhibit high variance with elevated refusal costs. Using elite prompts evolved on LLaMA 3.1 8B, we observe practically meaningful but attenuated cross-model transfer, with toxicity roughly halving on most targets, smaller LLaMA 3.2 variants showing the strongest resistance, and some cross-architecture models retaining higher toxicity. These results suggest that small, controllable perturbations are effective vehicles for systematic red-teaming and that defenses should anticipate cross-model reuse of adversarial prompts rather than focusing only on single-model hardening.

翻译：即使经过安全对齐，大型语言模型仍易受对抗性提示词影响而生成有害内容。本文提出ToxSearch——一种黑盒演化框架，通过在同步稳态循环中演化提示词来测试模型安全性。该系统采用多样化操作算子，包括词汇替换、否定、回译、复述及两种语义交叉算子，同时通过审核预言机提供适应度指导。算子级分析显示异质性行为：词汇替换在产出率与方差间达到最佳平衡，语义相似性交叉算子作为精确的低通量插入器，全局重写则表现出高方差并伴随更高的拒绝成本。使用在LLaMA 3.1 8B上演化出的精英提示词进行测试，我们观察到具有实际意义但有所衰减的跨模型迁移效应：多数目标模型的毒性输出约减少一半，较小规模的LLaMA 3.2变体表现出最强的抵抗能力，而部分跨架构模型则保留较高毒性。这些结果表明，微小可控的扰动是系统性红队测试的有效载体，防御机制应当预见到对抗性提示词的跨模型复用风险，而非仅关注单一模型的加固。