With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples. It has been a new trend to explore ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress and challenges of ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques, including training strategies, demonstration designing strategies, as well as related analysis. Finally, we discuss the challenges of ICL and provide potential directions for further research. We hope that our work can encourage more research on uncovering how ICL works and improving ICL.

翻译：随着大型语言模型（LLMs）能力的不断增强，上下文学习（ICL）已成为自然语言处理（NLP）领域的一种新范式，其中LLMs仅基于少量示例增强的上下文进行预测。探索ICL以评估和推断LLMs的能力已成为一种新趋势。本文旨在综述并总结ICL的进展与挑战。我们首先给出ICL的形式化定义，并阐明其与相关研究的关联。随后，我们组织并讨论了先进技术，包括训练策略、示例设计策略及相关分析。最后，我们探讨了ICL面临的挑战，并提出了潜在的研究方向。希望我们的工作能激励更多研究揭示ICL的工作原理并改进ICL。