Unrestricted adversarial attacks typically manipulate the semantic content of an image (e.g., color or texture) to create adversarial examples that are both effective and photorealistic. Recent works have utilized the diffusion inversion process to map images into a latent space, where high-level semantics are manipulated by introducing perturbations. However, they often results in substantial semantic distortions in the denoised output and suffers from low efficiency. In this study, we propose a novel framework called Semantic-Consistent Unrestricted Adversarial Attacks (SCA), which employs an inversion method to extract edit-friendly noise maps and utilizes Multimodal Large Language Model (MLLM) to provide semantic guidance throughout the process. Under the condition of rich semantic information provided by MLLM, we perform the DDPM denoising process of each step using a series of edit-friendly noise maps, and leverage DPM Solver++ to accelerate this process, enabling efficient sampling with semantic consistency. Compared to existing methods, our framework enables the efficient generation of adversarial examples that exhibit minimal discernible semantic changes. Consequently, we for the first time introduce Semantic-Consistent Adversarial Examples (SCAE). Extensive experiments and visualizations have demonstrated the high efficiency of SCA, particularly in being on average 12 times faster than the state-of-the-art attacks. Our code can be found at https://github.com/Pan-Zihao/SCA.

翻译：无限制对抗攻击通常通过操纵图像的语义内容（如颜色或纹理）来创建既有效又逼真的对抗样本。近期研究利用扩散反演过程将图像映射到潜在空间，并通过引入扰动来操纵高级语义。然而，这些方法往往导致去噪输出产生显著的语义失真，且效率低下。在本研究中，我们提出了一种名为语义一致无限制对抗攻击（SCA）的新框架，该框架采用反演方法提取易于编辑的噪声图，并利用多模态大语言模型（MLLM）在整个过程中提供语义指导。在MLLM提供丰富语义信息的条件下，我们使用一系列易于编辑的噪声图执行每一步的DDPM去噪过程，并利用DPM Solver++加速该过程，从而实现具有语义一致性的高效采样。与现有方法相比，我们的框架能够高效生成仅表现出极小可察觉语义变化的对抗样本。因此，我们首次引入了语义一致对抗样本（SCAE）。大量实验和可视化结果证明了SCA的高效性，其平均速度比最先进的攻击方法快12倍。我们的代码可在https://github.com/Pan-Zihao/SCA获取。