Autoregressive large language models (LLMs) have achieved remarkable success in many complex tasks, yet they can still fail in very simple logical reasoning such as the "reversal curse" -- when trained on forward knowledge data of the form "$A \rightarrow B$" (e.g., Alice's husband is Bob), the model is unable to deduce the reversal knowledge "$B \leftarrow A$" (e.g., Bob's wife is Alice) during test. Extensive prior research suggests that this failure is an inherent, fundamental limit of autoregressive causal LLMs, indicating that these models tend to memorize factual-level knowledge rather than capture higher-level rules. In this paper, we challenge this view by showing that this seemingly fundamental limit can be mitigated by slightly tweaking the training data with a simple regularization data recipe called the Identity Bridge of the form "$A \to A$" (e.g., The name of Alice is Alice). Theoretically, we prove that under this recipe, even a one-layer transformer can break the reversal curse by analyzing the implicit bias of gradient descent. Empirically, we show that a 1B pretrained language model finetuned with the proposed data recipe achieves a 40% success rate on reversal tasks, in stark contrast to a near-zero success rate when trained solely on forward-knowledge data. Our work provides a novel theoretical foundation for the reversal curse and offers a principled, low-cost path to encouraging LLMs to learn higher-level rules from data.

翻译：自回归大语言模型（LLMs）在众多复杂任务中取得了显著成功，但在某些极其简单的逻辑推理中仍会失败，例如“逆转诅咒”——当模型在形式为“$A \rightarrow B$”（例如，Alice的丈夫是Bob）的前向知识数据上训练后，在测试时无法推导出逆向知识“$B \leftarrow A$”（例如，Bob的妻子是Alice）。大量先前研究表明，这种失败是自回归因果LLMs固有的、根本性的局限，表明这些模型倾向于记忆事实层面的知识，而非捕捉更高层次的规则。本文挑战了这一观点，通过证明这种看似根本的局限可以通过对训练数据进行微调来缓解，具体采用一种称为“身份桥”的简单正则化数据方案，其形式为“$A \to A$”（例如，Alice的名字是Alice）。理论上，我们通过分析梯度下降的隐式偏差，证明了在此方案下，即使单层Transformer也能突破逆转诅咒。实证上，我们展示了采用所提数据方案微调的1B预训练语言模型在逆转任务上取得了40%的成功率，与仅在前向知识数据上训练时接近零的成功率形成鲜明对比。我们的工作为逆转诅咒提供了新的理论基础，并为鼓励LLMs从数据中学习更高层次规则提供了一条原则性、低成本的路径。