Large Language Models (LLMs) have shown promising in-context learning abilities. However, conventional In-Context Learning (ICL) approaches are often impeded by length limitations of transformer architecture, which pose challenges when attempting to effectively integrate supervision from a substantial number of demonstration examples. In this paper, we introduce a novel framework, called Naive Bayes-based Context Extension (NBCE), to enable existing LLMs to perform ICL with an increased number of demonstrations by significantly expanding their context size. Importantly, this expansion does not require fine-tuning or dependence on particular model architectures, all the while preserving linear efficiency. NBCE initially splits the context into equal-sized windows fitting the target LLM's maximum length. Then, it introduces a voting mechanism to select the most relevant window, regarded as the posterior context. Finally, it employs Bayes' theorem to generate the test task. Our experimental results demonstrate that NBCE substantially enhances performance, particularly as the number of demonstration examples increases, consistently outperforming alternative methods. The NBCE code will be made publicly accessible. The code NBCE is available at: https://github.com/amurtadha/NBCE-master

翻译：大语言模型（LLMs）已展现出显著的上下文学习能力。然而，传统的上下文学习方法常受限于Transformer架构的长度限制，这在尝试有效整合大量示范样例的监督信息时带来了挑战。本文提出一种名为"基于朴素贝叶斯的上下文扩展"（NBCE）的新型框架，通过显著扩展上下文规模，使现有大语言模型能够利用更多示范样例进行上下文学习。重要的是，这种扩展无需微调或依赖特定模型架构，同时保持线性效率。NBCE首先将上下文分割成与目标大语言模型最大长度适配的等宽窗口，随后引入投票机制选择最相关的窗口作为后验上下文，最终利用贝叶斯定理生成测试任务。实验结果表明，NBCE显著提升了模型性能，特别是在示范样例数量增加时，始终优于其他方法。NBCE代码将公开发布，访问地址：https://github.com/amurtadha/NBCE-master