Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, particularly in solving complex mathematical problems. Recent studies show that distilling long reasoning trajectories can effectively enhance the reasoning performance of small-scale student models. However, teacher-generated reasoning trajectories are often excessively long and structurally complex, making them difficult for student models to learn. This mismatch leads to a gap between the provided supervision signal and the learning capacity of the student model. To address this challenge, we propose Prefix-ALIGNment distillation (P-ALIGN), a framework that fully exploits teacher CoTs for distillation through adaptive prefix alignment. Specifically, P-ALIGN adaptively truncates teacher-generated reasoning trajectories by determining whether the remaining suffix is concise and sufficient to guide the student model. Then, P-ALIGN leverages the teacher-generated prefix to supervise the student model, encouraging effective prefix alignment. Experiments on multiple mathematical reasoning benchmarks demonstrate that P-ALIGN outperforms all baselines by over 3%. Further analysis indicates that the prefixes constructed by P-ALIGN provide more effective supervision signals, while avoiding the negative impact of redundant and uncertain reasoning components. All code is available at https://github.com/NEUIR/P-ALIGN.

翻译：大语言模型（LLMs）在解决复杂数学问题等任务中展现出卓越的推理能力。近期研究表明，通过蒸馏长推理轨迹可有效提升小规模学生模型的推理性能。然而，教师模型生成的推理轨迹往往过长且结构复杂，导致学生模型难以有效学习。这种不匹配使得教师提供的监督信号与学生模型的学习能力之间存在差距。为应对这一挑战，本文提出前缀对齐蒸馏框架（P-ALIGN），该框架通过自适应前缀对齐机制，充分利用教师模型的思维链进行知识蒸馏。具体而言，P-ALIGN通过判断剩余后缀是否简洁且足以指导学生模型，自适应截断教师生成的推理轨迹。随后，P-ALIGN利用教师生成的前缀监督学生模型，促进有效的前缀对齐。在多个数学推理基准测试上的实验表明，P-ALIGN以超过3%的优势优于所有基线方法。进一步分析显示，P-ALIGN构建的前缀能提供更有效的监督信号，同时避免冗余及不确定推理成分带来的负面影响。所有代码已开源：https://github.com/NEUIR/P-ALIGN。