Neural networks often favor shortcut heuristics based on surface-level patterns. As one example, language models (LMs) behave like n-gram models early in training. However, to correctly apply grammatical rules, LMs must rely on hierarchical syntactic representations instead of n-grams. In this work, we use cases studies of English grammar to explore how latent structure in training data drives models toward improved out-of-distribution (OOD) generalization.We then investigate how data composition can lead to inconsistent OOD behavior across random seeds and to unstable training dynamics. Our results show that models stabilize in their OOD behavior only when they fully commit to either a surface-level linear rule or a hierarchical rule. The hierarchical rule, furthermore, is induced by grammatically complex sequences with deep embedding structures, whereas the linear rule is induced by simpler sequences. When the data contains a mix of simple and complex examples, potential rules compete; each independent training run either stabilizes by committing to a single rule or remains unstable in its OOD behavior. These conditions lead `stable seeds' to cluster around simple rules, forming bimodal performance distributions across seeds. We also identify an exception to the relationship between stability and generalization: models which memorize patterns from low-diversity training data can overfit stably, with different rules for memorized and unmemorized patterns. Our findings emphasize the critical role of training data in shaping generalization patterns and how competition between data subsets contributes to inconsistent generalization outcomes across random seeds. Code is available at https://github.com/sunnytqin/concept_comp.git.

翻译：神经网络往往偏好基于表层模式的捷径启发式方法。例如，语言模型在训练初期会表现出类似n-gram模型的行为。然而，要正确应用语法规则，语言模型必须依赖层次化的句法表征而非n-gram。本研究通过英语语法的案例研究，探讨训练数据中的潜在结构如何驱动模型实现更好的分布外泛化。我们进一步研究数据构成如何导致不同随机种子间不一致的分布外行为及不稳定的训练动态。结果表明，仅当模型完全遵循表层线性规则或层次化规则时，其分布外行为才会稳定。层次化规则由具有深层嵌套结构的复杂语法序列诱导产生，而线性规则则由简单序列诱导。当数据混合包含简单与复杂样本时，潜在规则相互竞争；每次独立训练要么通过遵循单一规则达到稳定，要么在分布外行为上保持不稳定。这些条件导致"稳定种子"围绕简单规则聚集，形成跨种子的双峰性能分布。我们还发现了稳定性与泛化关系的一个例外：对低多样性训练数据模式进行记忆的模型能够稳定过拟合，并对已记忆和未记忆模式采用不同规则。本研究结果强调了训练数据在塑造泛化模式中的关键作用，以及数据子集间的竞争如何导致随机种子间不一致的泛化结果。代码发布于https://github.com/sunnytqin/concept_comp.git。