Post-training fundamentally alters the behavior of large language models (LLMs), yet its impact on the internal parameter space remains poorly understood. In this work, we conduct a systematic singular value decomposition (SVD) analysis of principal linear layers in pretrained LLMs, focusing on two widely adopted post-training methods: instruction tuning and long-chain-of-thought (Long-CoT) distillation. Our analysis reveals two unexpected and robust structural changes: (1) a near-uniform geometric scaling of singular values across layers; and (2) highly consistent orthogonal transformations are applied to the left and right singular vectors of each matrix. Based on these findings, We propose a simple yet effective framework to describe the coordinated dynamics of parameters in LLMs, which elucidates why post-training inherently relies on the foundational capabilities developed during pre-training. Further experiments demonstrate that singular value scaling underpins the temperature-controlled regulatory mechanisms of post-training, while the coordinated rotation of singular vectors encodes the essential semantic alignment. These results challenge the prevailing view of the parameter space in large models as a black box, uncovering the first clear regularities in how parameters evolve during training, and providing a new perspective for deeper investigation into model parameter changes.

翻译：训练后处理从根本上改变了大型语言模型（LLMs）的行为，然而其对内部参数空间的影响仍知之甚少。本研究对预训练LLMs中主要线性层进行了系统的奇异值分解（SVD）分析，重点关注两种广泛采用的训练后方法：指令微调和长思维链（Long-CoT）蒸馏。我们的分析揭示了两个出乎意料且稳健的结构变化：（1）各层奇异值呈现出近乎均匀的几何缩放；（2）每个矩阵的左、右奇异向量均施加了高度一致的正交变换。基于这些发现，我们提出了一个简单而有效的框架来描述LLMs中参数的协同动态，该框架阐明了为何训练后处理本质上依赖于预训练阶段所建立的基础能力。进一步的实验表明，奇异值缩放是训练后处理中温度调控机制的基础，而奇异向量的协同旋转则编码了关键的语义对齐信息。这些结果挑战了当前将大模型参数空间视为黑箱的主流观点，首次揭示了参数在训练过程中如何演变的清晰规律，并为深入研究模型参数变化提供了新的视角。