Recent studies have demonstrated that large language models (LLMs) excel in diverse tasks through in-context learning (ICL) facilitated by task-specific prompts and examples. However, the existing literature shows that ICL encounters performance deterioration when exposed to adversarial inputs. Enhanced performance has been observed when ICL is augmented with natural language explanations (NLEs) (we refer to it as X-ICL). Thus, this work investigates whether X-ICL can improve the robustness of LLMs on a suite of seven adversarial and challenging natural language inference datasets. Moreover, we introduce a new approach to X-ICL by prompting an LLM (ChatGPT in our case) with few human-generated NLEs to produce further NLEs (we call it ChatGPT few-shot), which we show superior to both ChatGPT zero-shot and human-generated NLEs alone. We evaluate five popular LLMs (GPT3.5-turbo, LLaMa2, Vicuna, Zephyr, Mistral) and show that X-ICL with ChatGPT few-shot yields over 6% improvement over ICL. Furthermore, while prompt selection strategies were previously shown to significantly improve ICL on in-distribution test sets, we show that these strategies do not match the efficacy of the X-ICL paradigm in robustness-oriented evaluations.

翻译：近期研究表明，大语言模型通过任务特定提示和示例驱动的上下文学习，能够出色完成多样化任务。然而现有文献显示，当面对对抗性输入时，上下文学习会出现性能退化。当将自然语言解释增强到上下文学习中时（我们称之为X-ICL），模型性能得到提升。为此，本文探究X-ICL是否能在七个对抗性与挑战性自然语言推理数据集上提升大语言模型的鲁棒性。此外，我们提出一种新的X-ICL方法：通过用少量人工生成的自然语言解释提示目标模型（本文为ChatGPT）生成更多解释（称为ChatGPT小样本法），该方法的性能优于ChatGPT零样本法和仅使用人工生成的自然语言解释。我们对五种主流大语言模型（GPT3.5-turbo、LLaMa2、Vicuna、Zephyr、Mistral）进行评估，结果表明采用ChatGPT小样本法的X-ICL相比纯上下文学习性能提升超过6%。值得注意的是，尽管先前研究表明提示选择策略能显著提升上下文学习在分布内测试集上的表现，但本文证明在鲁棒性导向评估中，这些策略无法达到X-ICL范式同等效果。