The surprising ability of Large Language Models (LLMs) to perform well on complex reasoning with only few-shot chain-of-thought prompts is believed to emerge only in very large-scale models (100+ billion parameters). We show that such abilities can, in fact, be distilled down from GPT-3.5 ($\ge$ 175B) to T5 variants ($\le$ 11B). We propose model specialization, to specialize the model's ability towards a target task. The hypothesis is that large models (commonly viewed as larger than 100B) have strong modeling power, but are spread on a large spectrum of tasks. Small models (commonly viewed as smaller than 10B) have limited model capacity, but if we concentrate their capacity on a specific target task, the model can achieve a decent improved performance. We use multi-step math reasoning as our testbed because it is a very typical emergent ability. We show two important aspects of model abilities: (1). there exists a very complex balance/ tradeoff between language models' multi-dimensional abilities; (2). by paying the price of decreased generic ability, we can clearly lift up the scaling curve of models smaller than 10B towards a specialized multi-step math reasoning ability. We further give comprehensive discussions about important design choices for better generalization, including the tuning data format, the start model checkpoint, and a new model selection method. We hope our practice and discoveries can serve as an important attempt towards specialized smaller models in the new research paradigm set by LLMs.

翻译：大型语言模型（LLMs）在仅需少量链式思维提示即可在复杂推理任务中表现出色的惊人能力，被认为仅在超大规模模型（1000亿以上参数）中才会涌现。我们证明，这种能力实际上可以从GPT-3.5（≥1750亿参数）蒸馏至T5变体模型（≤110亿参数）。我们提出模型专业化方法，使模型专注于特定目标任务。其核心假设是：大模型（通常指超过1000亿参数）虽具有强大建模能力，但其能力分散在广泛的任务谱系中；而小模型（通常指低于100亿参数）虽然模型容量有限，但若将其能力集中用于特定目标任务，仍能实现显著的性能提升。我们以多步数学推理作为测试场景，因其属于典型的涌现能力。本研究揭示模型能力的两个重要特性：(1) 语言模型的多维能力存在极为复杂的平衡/权衡关系；(2) 通过牺牲泛化能力为代价，可显著提升低于100亿参数模型在专业化多步数学推理上的规模缩放曲线。我们进一步系统讨论了提升泛化性能的关键设计选择，包括微调数据格式、初始模型检查点选择，以及新型模型筛选方法。期望本研究的实践与发现，能为大语言模型所开创的新研究范式下的小型专业化模型探索，提供重要参考案例。