While most existing works on LLM prompt-engineering focus only on how to select a better set of data samples inside one single prompt input (In-Context Learning or ICL), why can't we design and leverage multiple prompt inputs together to further improve the LLM performance? In this work, we propose In-Context Sampling (ICS), a low-resource LLM prompt-engineering technique to produce the most confident prediction results by optimizing the construction of multiple ICL prompt inputs. Extensive experiments with two SOTA LLMs (FlanT5-XL and Mistral-7B) on three NLI datasets (e-SNLI, Multi-NLI, and ANLI) illustrate that ICS can consistently enhance LLM's prediction performance and confidence. An ablation study suggests that a diversity-based ICS strategy may further improve LLM's performance, which sheds light on a new yet promising future research direction.

翻译：尽管现有关于大语言模型提示工程的研究大多仅关注如何在一个提示输入内选择更优的数据样本集（上下文学习），但我们为何不能设计并利用多个提示输入共同进一步优化大语言模型性能？本研究提出上下文采样方法，这是一种低资源的大语言模型提示工程技术，通过优化多个上下文学习提示输入的构建方式，生成最具置信度的预测结果。在三个自然语言推理数据集（e-SNLI、Multi-NLI和ANLI）上，使用两个当前最优大语言模型（FlanT5-XL和Mistral-7B）进行的广泛实验表明，ICS能持续提升大语言模型的预测性能与置信度。消融实验发现，基于多样性的ICS策略可进一步优化大语言模型性能，这为未来研究开辟了一个新颖且有前景的方向。