The "reversal curse" refers to the phenomenon where large language models (LLMs) exhibit predominantly unidirectional behavior when processing logically bidirectional relationships. Prior work attributed this to autoregressive training -- predicting the next token inherently favors left-to-right information flow over genuine bidirectional knowledge associations. However, we observe that Diffusion LLMs (DLLMs), despite being trained bidirectionally, also suffer from the reversal curse. To investigate the root causes, we conduct systematic experiments on DLLMs and identify three key reasons: 1) entity fragmentation during training, 2) data asymmetry, and 3) missing entity relations. Motivated by the analysis of these reasons, we propose Diffusion Entity-Relation Modeling (DiffER), which addresses the reversal curse through entity-aware training and balanced data construction. Specifically, DiffER introduces whole-entity masking, which mitigates entity fragmentation by predicting complete entities in a single step. DiffER further employs distribution-symmetric and relation-enhanced data construction strategies to alleviate data asymmetry and missing relations. Extensive experiments demonstrate that DiffER effectively alleviates the reversal curse in Diffusion LLMs, offering new perspectives for future research.

翻译：“逆向诅咒”指大语言模型在处理逻辑双向关系时主要表现出单向行为的现象。先前研究将其归因于自回归训练——预测下一词元本质上更倾向于从左到右的信息流，而非真正的双向知识关联。然而，我们观察到扩散大语言模型尽管经过双向训练，同样受到逆向诅咒的影响。为探究根本原因，我们对扩散大语言模型进行了系统实验，并识别出三个关键因素：1）训练过程中的实体碎片化，2）数据不对称性，3）缺失的实体关系。基于对这些原因的分析，我们提出扩散实体关系建模方法，通过实体感知训练与均衡数据构建来解决逆向诅咒问题。具体而言，DiffER引入整体实体掩码机制，通过单步预测完整实体来缓解实体碎片化。DiffER进一步采用分布对称与关系增强的数据构建策略，以减轻数据不对称性和关系缺失问题。大量实验表明，DiffER能有效缓解扩散大语言模型中的逆向诅咒，为未来研究提供了新的视角。