Large language models (LLMs) with enormous pre-training tokens and parameter amounts emerge abilities, including math reasoning, code generation, and instruction following. These abilities are further enhanced by supervised fine-tuning (SFT). The open-source community has studied on ad-hoc SFT for each ability, while proprietary LLMs are versatile for all abilities. It is important to investigate how to unlock them with multiple abilities via SFT. In this study, we specifically focus on the data composition between mathematical reasoning, code generation, and general human-aligning abilities during SFT. From a scaling perspective, we investigate the relationship between model abilities and various factors including data amounts, data composition ratio, model parameters, and SFT strategies. Our experiments reveal that different abilities exhibit different scaling patterns, and larger models generally show superior performance with the same amount of data. Mathematical reasoning and code generation improve as data amounts increase consistently, while the general ability is enhanced with about a thousand samples and improves slowly. We find data composition results in various abilities improvements with low data amounts, while conflicts of abilities with high data amounts. Our experiments further show that composition data amount impacts performance, while the influence of composition ratio is insignificant. Regarding the SFT strategies, we evaluate sequential learning multiple abilities are prone to catastrophic forgetting. Our proposed Dual-stage Mixed Fine-tuning (DMT) strategy learns specialized abilities first and then learns general abilities with a small amount of specialized data to prevent forgetting, offering a promising solution to learn multiple abilities with different scaling patterns.

翻译：大规模语言模型（LLMs）凭借海量预训练数据和参数规模，涌现出数学推理、代码生成和指令遵循等能力。这些能力通过监督微调（SFT）得到进一步增强。开源社区针对单一能力进行了定制化SFT研究，而商用大模型则具备多能力的通用性。探究如何通过SFT同时解锁多项能力具有重要意义。本研究聚焦监督微调过程中数学推理、代码生成与通用人类对齐能力之间的数据构成关系。从规模化视角出发，我们研究了模型能力与数据量、数据构成比例、模型参数及SFT策略等要素的关联性。实验表明，不同能力呈现不同的扩展规律：在相同数据量下，更大规模的模型通常表现更优；数学推理与代码生成能力随数据量增加持续提升，而通用能力在约千条样本后提升趋缓。研究发现低数据量时数据构成能促进各项能力提升，而高数据量时则会出现能力冲突。进一步实验表明，数据构成总量影响模型表现，但构成比例的影响不显著。在SFT策略方面，顺序学习多项能力易导致灾难性遗忘。我们提出的双阶段混合微调（DMT）策略先学习专项能力，再通过少量专项数据学习通用能力以防遗忘，为学习具有不同扩展规律的多项能力提供了有效方案。