Recent studies have highlighted a phenomenon in large language models (LLMs) known as "the reversal curse," in which the order of knowledge entities in the training data biases the models' comprehension. For example, if a model is trained on sentences where entity A consistently appears before entity B, it can respond to queries about A by providing B as the answer. However, it may encounter confusion when presented with questions concerning B. We contend that the reversal curse is partially a result of specific model training objectives, particularly evident in the prevalent use of the next-token prediction within most causal language models. For the next-token prediction, models solely focus on a token's preceding context, resulting in a restricted comprehension of the input. In contrast, we illustrate that the GLM, trained using the autoregressive blank infilling objective where tokens to be predicted have access to the entire context, exhibits better resilience against the reversal curse. We propose a novel training method, BIdirectional Casual language modeling Optimization (BICO), designed to mitigate the reversal curse when fine-tuning pretrained causal language models on new data. BICO modifies the causal attention mechanism to function bidirectionally and employs a mask denoising optimization. In the task designed to assess the reversal curse, our approach improves Llama's accuracy from the original 0% to around 70%. We hope that more attention can be focused on exploring and addressing these inherent weaknesses of the current LLMs, in order to achieve a higher level of intelligence.

翻译：近期研究揭示了大语言模型（LLMs）中存在一种被称为“逆转诅咒”的现象，即训练数据中知识实体的顺序会影响模型的理解能力。例如，若模型在训练中反复遇到实体A先于实体B出现的句子，它能在回答关于A的问题时给出B作为答案，但在面对涉及B的提问时却可能产生困惑。我们认为，逆转诅咒部分源于特定的模型训练目标，这在大多数因果语言模型普遍采用的“下一词预测”任务中尤为突出。对于下一词预测，模型仅关注当前词之前的上下文，导致对输入信息的理解受限。相比之下，我们证明采用自回归空白填充训练目标的GLM模型（其预测词可访问完整上下文）能更好地抵御逆转诅咒。我们提出了一种新型训练方法——双向因果语言建模优化（BICO），旨在缓解预训练因果语言模型在新数据微调过程中的逆转诅咒效应。BICO方法通过改造因果注意力机制使其双向运作，并采用掩码去噪优化策略。在专门设计的逆转诅咒评估任务中，我们的方法将Llama模型的准确率从原始0%提升至约70%。我们期待更多研究关注并解决当前LLMs的这些固有缺陷，以实现更高水平的智能。