While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the "reversal curse". It is a typical example that the model knows "A's father is B", but is unable to reason "B's child is A". This limitation poses a challenge to the advancement of artificial general intelligence (AGI), as it suggests a gap in the models' ability to comprehend and apply bidirectional reasoning. In this paper, we first conduct substantial evaluation and identify that the root cause of the reversal curse lies in the different word order between the training and inference stage, namely, the poor ability of causal language models to predict antecedent words within the training data. Accordingly, permutation on the training data is considered as a potential solution, since this can make the model predict antecedent words or tokens. However, previous permutation methods may disrupt complete phrases or entities, thereby posing challenges for the model to comprehend and learn from training data. To address this issue, we propose Semantic-aware Permutation Training (SPT), which addresses this issue by segmenting the training sentences into semantic units (i.e., entities or phrases) with an assistant language model and permuting these units before feeding into the model. Extensive experiments demonstrate that SPT effectively mitigates the reversal curse since the performance on reversed questions approximates that on the forward ones, and significantly advances the performance of existing works.

翻译：尽管大语言模型（LLMs）已在各类任务中展现出卓越性能，但最新研究表明，因果大语言模型存在"反向诅咒"现象。典型例子是：模型知道"A的父亲是B"，却无法推理出"B的孩子是A"。这一局限性对通用人工智能（AGI）的发展构成挑战，因为它揭示了模型在理解和应用双向推理能力方面存在差距。本文首先通过系统评估，揭示了反向诅咒的根本原因在于训练与推理阶段词序的差异——即因果语言模型对训练数据中前文词汇的预测能力不足。据此，对训练数据进行置换被视为潜在解决方案，因为这能使模型预测前文词汇或标记。然而，现有置换方法可能破坏完整短语或实体，导致模型难以理解并学习训练数据。为解决该问题，我们提出语义感知置换训练（SPT），通过辅助语言模型将训练句子切分为语义单元（即实体或短语），并在输入模型前对这些单元进行置换。大量实验表明，SPT能有效缓解反向诅咒——模型在反向问题上的表现已接近正向问题，且显著提升了现有方法的性能。