With the increasing ability of large language models (LLMs), in-context learning (ICL) has become a new paradigm for natural language processing (NLP), where LLMs make predictions only based on contexts augmented with a few examples. It has been a new trend to explore ICL to evaluate and extrapolate the ability of LLMs. In this paper, we aim to survey and summarize the progress and challenges of ICL. We first present a formal definition of ICL and clarify its correlation to related studies. Then, we organize and discuss advanced techniques, including training strategies, demonstration designing strategies, as well as related analysis. Finally, we discuss the challenges of ICL and provide potential directions for further research. We hope that our work can encourage more research on uncovering how ICL works and improving ICL.

翻译：随着大型语言模型（LLMs）能力的不断增强，上下文学习（ICL）已成为自然语言处理（NLP）领域的一种新范式，其中LLMs仅基于包含少量示例的上下文进行预测。探索ICL以评估和推断LLMs的能力已成为新趋势。本文旨在综述并总结ICL的进展与挑战。我们首先给出ICL的形式化定义，并阐明其与相关研究的关联。随后，我们组织并讨论了先进技术，包括训练策略、示例设计策略及相关分析。最后，我们探讨了ICL面临的挑战，并提出了未来研究的潜在方向。我们希望这项工作能推动更多关于ICL工作机制及性能改进的研究。