Recent advances in deep learning research have shown remarkable achievements across many tasks in computer vision (CV) and natural language processing (NLP). At the intersection of CV and NLP is the problem of image captioning, where the related models' robustness against adversarial attacks has not been well studied. In this paper, we present a novel adversarial attack strategy, which we call AICAttack (Attention-based Image Captioning Attack), designed to attack image captioning models through subtle perturbations on images. Operating within a black-box attack scenario, our algorithm requires no access to the target model's architecture, parameters, or gradient information. We introduce an attention-based candidate selection mechanism that identifies the optimal pixels to attack, followed by Differential Evolution (DE) for perturbing pixels' RGB values. We demonstrate AICAttack's effectiveness through extensive experiments on benchmark datasets with multiple victim models. The experimental results demonstrate that our method surpasses current leading-edge techniques by effectively distributing the alignment and semantics of words in the output.

翻译：近期深度学习研究在计算机视觉（CV）和自然语言处理（NLP）的诸多任务中展现了显著成果。在CV与NLP的交叉领域——图像描述任务中，相关模型对抗性攻击的鲁棒性尚未得到充分研究。本文提出一种名为AICAttack（基于注意力的图像描述攻击）的新型对抗性攻击策略，旨在通过对图像施加细微扰动来攻击图像描述模型。该算法在黑盒攻击场景下运行，无需获取目标模型的架构、参数或梯度信息。我们引入基于注意力的候选像素选择机制，用于识别最优攻击像素，随后采用差分进化（DE）算法扰动像素的RGB值。通过在基准数据集上对多种受害模型进行大量实验，我们验证了AICAttack的有效性。实验结果表明，本方法通过有效破坏输出文本中词语的对齐性与语义，超越了当前前沿技术。