In-context learning (ICL) is a new paradigm for natural language processing (NLP), where a large language model (LLM) observes a small number of demonstrations and a test instance as its input, and directly makes predictions without updating model parameters. Previous studies have revealed that ICL is sensitive to the selection and the ordering of demonstrations. However, there are few studies regarding the impact of the demonstration number on the ICL performance within a limited input length of LLM, because it is commonly believed that the number of demonstrations is positively correlated with model performance. In this paper, we found this conclusion does not always hold true. Through pilot experiments, we discover that increasing the number of demonstrations does not necessarily lead to improved performance. Building upon this insight, we propose a Dynamic Demonstrations Controller (D$^2$Controller), which can improve the ICL performance by adjusting the number of demonstrations dynamically. The experimental results show that D$^2$Controller yields a 4.6% relative improvement on ten different sizes of LLMs across ten datasets. Moreover, we also extend our method to previous ICL models and achieve competitive results.

翻译：上下文学习（ICL）是自然语言处理（NLP）的一种新范式，其中大型语言模型（LLM）将少量演示示例和一个测试实例作为输入，直接进行预测而无需更新模型参数。先前的研究表明，ICL对演示示例的选择和排序非常敏感。然而，关于在LLM有限输入长度内演示数量对ICL性能影响的研究较少，因为通常认为演示数量与模型性能呈正相关。本文发现这一结论并不总是成立。通过初步实验，我们发现增加演示数量并不一定会带来性能提升。基于这一发现，我们提出了一种动态演示控制器（D$^2$Controller），它可以通过动态调整演示数量来提升ICL性能。实验结果表明，D$^2$Controller在十个数据集上针对十种不同规模的LLM实现了4.6%的相对性能提升。此外，我们将该方法扩展到先前的ICL模型中，并取得了具有竞争力的结果。