Various recent experimental results show that large language models (LLM) exhibit emergent abilities that are not present in small models. System performance is greatly improved after passing a certain critical threshold of scale. In this letter, we provide a simple explanation for such a phase transition phenomenon. For this, we model an LLM as a sequence-to-sequence random function. Instead of using instant generation at each step, we use a list decoder that keeps a list of candidate sequences at each step and defers the generation of the output sequence at the end. We show that there is a critical threshold such that the expected number of erroneous candidate sequences remains bounded when an LLM is below the threshold, and it grows exponentially when an LLM is above the threshold. Such a threshold is related to the basic reproduction number in a contagious disease.

翻译：近期多项实验结果表明，大型语言模型（LLM）展现出小规模模型所不具备的涌现能力。当模型规模超过特定临界阈值后，系统性能显著提升。本文针对这种相变现象提出一个简洁解释。我们将大语言模型建模为序列到序列的随机函数，采用在每个步骤保留候选序列列表的列表解码器替代即时逐次生成，并在最终步骤输出序列。研究表明存在一个临界阈值：当模型规模低于该阈值时，错误候选序列的期望数量保持有界；当模型规模超过该阈值时，错误候选序列数量呈指数增长。该阈值与传染病领域的基本再生数具有关联性。