On-policy distillation (\textsc{OPD}) has recently become a prominent post-training recipe as it combines two desirable ingredients: on-policy student trajectories and dense teacher supervision, yet how this hybrid changes a model's parameters remains unclear. Across several language and vision-language model pairs and use cases, our analysis yields two main findings. On sparsity, \textsc{OPD}-style updates are small and coordinate-sparse. They are distributed across layers and are usually FFN-heavy. This sparse structure is operationally useful: training only the discovered subnetwork recovers nearly the same performance as full \textsc{OPD}. However, the sparsity-inducing SGD optimizer underperforms AdamW in our optimizer ablation, likely because dense teacher supervision preserves heterogeneous coordinate-wise gradient scales where AdamW's adaptive scaling remains useful. On geometry, the updates are numerically full-rank but spectrally concentrated; they lie mostly away from the principal singular subspaces of the source weights and fall disproportionately on coordinates where the source weights are close to zero. These findings suggest that dense teacher supervision does not turn \textsc{OPD} into ordinary dense parameter rewriting; instead, \textsc{OPD} retains important geometric signatures of on-policy post-training.

翻译：在线策略蒸馏（On-policy Distillation，\textsc{OPD}）近期已成为一种重要的后训练方法，因为它融合了两个理想要素：在线策略的学生轨迹与密集的教师监督，然而这种混合方式如何改变模型参数尚不明确。通过对多个语言模型及视觉-语言模型对与用例的分析，我们得出两个主要发现。在稀疏性方面，\textsc{OPD}风格的更新幅度小且坐标稀疏，这些更新分布在各层中，通常以前馈网络（FFN）为主。这种稀疏结构具有实际效用：仅训练所发现的子网络即可恢复近乎完整的\textsc{OPD}性能。然而，在我们的优化器消融实验中，诱导稀疏性的SGD优化器性能逊于AdamW，这很可能是因为密集的教师监督保留了异质化的坐标方向梯度尺度，而AdamW的自适应缩放能力在此情境下仍具价值。在几何特性方面，这些更新在数值上满秩但谱域集中，主要偏离源权重的奇异主子空间，且不均衡地作用于源权重接近零的坐标上。这些发现表明，密集的教师监督并未将\textsc{OPD}转变为普通的密集参数重写；相反，\textsc{OPD}保留了在线策略后训练的重要几何特征。