Recent studies have highlighted a phenomenon in large language models (LLMs) known as "the reversal curse," in which the order of knowledge entities in the training data biases the models' comprehension. For example, if a model is trained on sentences where entity A consistently appears before entity B, it can respond to queries about A by providing B. However, it may encounter confusion when presented with questions concerning B. We contend that the reversal curse is partially a result of specific model training objectives, particularly evident in the prevalent use of the next-token prediction within most causal language models. For the next-token prediction, models solely focus on a token's preceding context, resulting in a restricted comprehension of the input. In contrast, we illustrate that the GLM, trained using the autoregressive blank infilling objective where tokens to be predicted have access to the entire context, exhibits better resilience against the reversal curse. We propose a novel training method, BIdirectional Casual language modeling Optimization (BICO), designed to mitigate the reversal curse when fine-tuning pretrained causal language models on new data. BICO modifies the causal attention mechanism to function bidirectionally and employs a mask denoising optimization. In the task designed to assess the reversal curse, our approach improves Llama's accuracy from the original 0% to around 70%. We hope that more attention can be focused on exploring and addressing these inherent weaknesses of the current LLMs, in order to achieve a higher level of intelligence.

翻译：近期研究揭示了大型语言模型（LLMs）中存在一种称为"反转诅咒"的现象，即训练数据中知识实体的顺序会扭曲模型的理解能力。例如，若模型在实体A始终出现在实体B之前的语句上进行训练，它能正确回答关于A的查询并给出B，但当遇到关于B的问题时则可能产生混淆。我们认为，反转诅咒部分源于特定模型训练目标，尤其体现在多数因果语言模型广泛采用的下一词元预测中。由于该机制仅关注词元的前置上下文，导致模型对输入的理解受限。相比之下，我们证明采用自回归空白填充训练目标（待预测词元可访问完整上下文）的GLM模型对反转诅咒具有更强的鲁棒性。我们提出新型训练方法——双向因果语言建模优化（BICO），旨在微调预训练因果语言模型时缓解反转诅咒。BICO将因果注意力机制改造为双向运作，并采用掩码去噪优化。在面向反转诅咒评估的任务中，我们的方法将Llama的准确率从初始的0%提升至约70%。我们期待更多研究聚焦于探索并解决当前LLMs的这些固有缺陷，以实现更高水平的智能。