With the emergence of large language models, such as LLaMA and OpenAI GPT-3, In-Context Learning (ICL) gained significant attention due to its effectiveness and efficiency. However, ICL is very sensitive to the choice, order, and verbaliser used to encode the demonstrations in the prompt. Retrieval-Augmented ICL methods try to address this problem by leveraging retrievers to extract semantically related examples as demonstrations. While this approach yields more accurate results, its robustness against various types of adversarial attacks, including perturbations on test samples, demonstrations, and retrieved data, remains under-explored. Our study reveals that retrieval-augmented models can enhance robustness against test sample attacks, outperforming vanilla ICL with a 4.87% reduction in Attack Success Rate (ASR); however, they exhibit overconfidence in the demonstrations, leading to a 2% increase in ASR for demonstration attacks. Adversarial training can help improve the robustness of ICL methods to adversarial attacks; however, such a training scheme can be too costly in the context of LLMs. As an alternative, we introduce an effective training-free adversarial defence method, DARD, which enriches the example pool with those attacked samples. We show that DARD yields improvements in performance and robustness, achieving a 15% reduction in ASR over the baselines. Code and data are released to encourage further research: https://github.com/simonucl/adv-retreival-icl

翻译：随着LLaMA和OpenAI GPT-3等大语言模型的出现，上下文学习因其高效性和有效性而受到广泛关注。然而，ICL对提示中示例的选择、排序以及用于编码示例的表述方式极为敏感。检索增强型ICL方法试图通过利用检索器提取语义相关的示例作为演示来解决这一问题。尽管这种方法能获得更准确的结果，但其针对各类对抗攻击（包括对测试样本、演示示例和检索数据的扰动）的鲁棒性仍未得到充分探索。我们的研究表明，检索增强模型能够提升对测试样本攻击的鲁棒性，其攻击成功率较原始ICL方法降低4.87%；然而，这类模型对演示示例表现出过度置信，导致在面对演示攻击时ASR上升2%。对抗性训练有助于提升ICL方法对对抗攻击的鲁棒性，但在大语言模型背景下，这种训练方案可能成本过高。作为替代方案，我们提出了一种有效的免训练对抗防御方法DARD，该方法通过引入受攻击样本来丰富示例池。我们证明DARD在性能和鲁棒性上均有提升，相较于基线方法实现了15%的ASR降低。代码和数据已开源以促进后续研究：https://github.com/simonucl/adv-retreival-icl