With the increasing capabilities of large language models (LLMs), in-context learning (ICL) has emerged as a new paradigm for natural language processing (NLP), where LLMs make predictions based on contexts augmented with a few examples. It has been a significant trend to explore ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress and challenges of ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques, including training strategies, prompt designing strategies, and related analysis. Additionally, we explore various ICL application scenarios, such as data engineering and knowledge updating. Finally, we address the challenges of ICL and suggest potential directions for further research. We hope that our work can encourage more research on uncovering how ICL works and improving ICL.

翻译：随着大型语言模型（LLM）能力的不断提升，上下文学习（ICL）已成为自然语言处理（NLP）的一种新范式，即LLM基于补充了少量示例的上下文进行预测。探索ICL以评估和推断LLM的能力已成为一个重要趋势。本文旨在综述并总结ICL的进展与挑战。我们首先给出ICL的形式化定义，并阐明其与相关研究的关联。随后，我们整理并讨论了包括训练策略、提示设计策略及相关分析在内的先进技术。此外，我们探讨了ICL的各种应用场景，例如数据工程与知识更新。最后，我们指出了ICL面临的挑战，并提出了未来研究的潜在方向。我们希望本工作能够鼓励更多研究，以揭示ICL的工作原理并改进ICL。