Deep neural networks are found to be vulnerable to adversarial noises. The prompt-based defense has been increasingly studied due to its high efficiency. However, existing prompt-based defenses mainly exploited mixed prompt patterns, where critical patterns closely related to object semantics lack sufficient focus. The phase and amplitude spectra have been proven to be highly related to specific semantic patterns and crucial for robustness. To this end, in this paper, we propose a Phase and Amplitude-aware Prompting (PAP) defense. Specifically, we construct phase-level and amplitude-level prompts for each class, and adjust weights for prompting according to the model's robust performance under these prompts during training. During testing, we select prompts for each image using its predicted label to obtain the prompted image, which is inputted to the model to get the final prediction. Experimental results demonstrate the effectiveness of our method.

翻译：深度神经网络被发现容易受到对抗性噪声的影响。基于提示的防御方法因其高效性而日益受到研究关注。然而，现有的基于提示的防御方法主要利用混合提示模式，其中与物体语义密切相关的关键模式缺乏足够关注。相位谱与幅度谱已被证明与特定语义模式高度相关，并对鲁棒性至关重要。为此，本文提出一种相位与幅度感知提示（PAP）防御方法。具体而言，我们为每个类别构建相位级与幅度级提示，并根据模型在这些提示下的训练鲁棒性表现调整提示权重。在测试阶段，我们使用图像的预测标签为其选择提示，获得提示增强后的图像，并将其输入模型以得到最终预测。实验结果证明了我们方法的有效性。