Understanding emergent abilities, such as in-context learning (ICL) and chain-of-thought (CoT) prompting in large language models (LLMs), is of utmost importance. This importance stems not only from the better utilization of these capabilities across various tasks, but also from the proactive identification and mitigation of potential risks, including concerns of truthfulness, bias, and toxicity, that may arise alongside these capabilities. In this paper, we present a thorough survey on the interpretation and analysis of emergent abilities of LLMs. First, we provide a concise introduction to the background and definition of emergent abilities. Then, we give an overview of advancements from two perspectives: 1) a macro perspective, emphasizing studies on the mechanistic interpretability and delving into the mathematical foundations behind emergent abilities; and 2) a micro-perspective, concerning studies that focus on empirical interpretability by examining factors associated with these abilities. We conclude by highlighting the challenges encountered and suggesting potential avenues for future research. We believe that our work establishes the basis for further exploration into the interpretation of emergent abilities.

翻译：理解涌现能力（如大语言模型中的上下文学习（ICL）和思维链（CoT）提示）至关重要。其重要性不仅在于能更好地将这些能力应用于各类任务，更在于能够主动识别并缓解这些能力可能伴随的潜在风险，包括真实性、偏见及毒性等问题。本文对涌现能力的诠释与分析进行了全面综述。首先，我们简要介绍涌现能力的背景与定义。随后，我们从两个视角概述研究进展：1) 宏观视角，聚焦于机制可解释性研究，深入探讨涌现能力背后的数学基础；2) 微观视角，关注通过检验与这些能力相关因素的实证可解释性研究。最后，我们总结了当前面临的挑战，并提出了未来研究的潜在方向。我们相信，本工作为深入探索涌现能力的诠释奠定了坚实基础。