Red-teaming Vision-Language Models is essential for identifying vulnerabilities where adversarial image-text inputs trigger toxic outputs. Existing approaches treat image generation as a black box, returning only terminal toxicity scores and leaving open the question of when and how toxic semantics emerge during multi-step synthesis. We introduce STARE, a hierarchical reinforcement learning framework that treats the denoising trajectory itself as the attack surface, under a direct white-box T2I and query-only black-box VLM setting. By coupling a high-level prompt editor with low-level T2I fine-tuning via Group Relative Policy Optimization (GRPO), STARE attains a 68\% improvement in Attack Success Rate over state-of-the-art black-box and white-box baselines. More importantly, this trajectory-level view surfaces the Optimization-Induced Phase Alignment phenomenon: vanilla models exhibit diffuse toxicity, whereas adversarial optimization concentrates conceptual harms into early semantic phases and detail-oriented harms into late refinement. Targeted perturbations of either window selectively suppress different toxicity categories, indicating that this temporal structure is a genuine causal handle rather than a side effect of the hierarchical design. The phenomenon turns toxicity formation from a chaotic process into a small set of predictable vulnerability windows, providing both a potent attack engine and a basis for phase-aware safety mechanisms. Content warning: This paper contains examples of toxic content that may be offensive or disturbing.

翻译：红队测试视觉-语言模型对于识别由对抗性图文输入引发毒性输出的漏洞至关重要。现有方法将图像生成视为黑盒，仅返回最终毒性分数，而未能回答在逐步合成过程中毒性语义何时及如何涌现的问题。我们提出STARE，一种层次化强化学习框架，将去噪轨迹本身视为攻击面，在直接白盒文本到图像（T2I）与仅查询的黑盒视觉-语言模型（VLM）设置下工作。通过将高层提示编辑器与基于组相对策略优化（GRPO）的低层T2I微调相结合，STARE在攻击成功率上相较于最先进的黑盒与白盒基线实现了68%的提升。更重要的是，这种轨迹级视角揭示了“优化诱导的相位对齐”现象：普通模型表现出弥散性毒性，而对抗优化将概念性危害集中于早期语义相位，将细节导向性危害集中于后期精炼相位。对任一时窗的定向扰动可选择性地抑制不同毒性类别，表明这种时间结构是一个真正的因果控制手段，而非层次化设计的副产品。该现象将毒性形成过程从混沌状态转化为一组可预测的漏洞时窗，既提供了强大的攻击引擎，也为相位感知的安全机制奠定了基础。内容警告：本文包含可能具有冒犯性或引发不适的毒性内容示例。