Despite the superior performance of large language models to generate natural language texts, it is hard to generate texts with correct logic according to a given task, due to the difficulties for neural models to capture implied rules from free-form texts. In this paper, we propose a novel graph-based language model, Logical-GLM, to infuse logic into language models for more valid text generation and interpretability. Specifically, we first capture information from natural language instructions and construct logical bayes graphs that generally describe domains. Next, we generate logical skeletons to guide language model training, infusing domain knowledge into language models. Finally, we alternately optimize the searching policy of graphs and language models until convergence. The experimental results show that Logical-GLM is both effective and efficient compared with traditional language models, despite using smaller-scale training data and fewer parameters. Our approach can generate instructional texts with more correct logic owing to the internalized domain knowledge. Moreover, the usage of logical graphs reflects the inner mechanism of the language models, which improves the interpretability of black-box models.

翻译：尽管大型语言模型在生成自然语言文本方面表现出卓越性能，但由于神经模型难以从自由文本中捕捉隐含规则，根据给定任务生成逻辑正确的文本仍然具有挑战性。本文提出一种新颖的基于图的语言模型Logical-GLM，通过将逻辑注入语言模型以实现更有效的文本生成和可解释性。具体而言，我们首先从自然语言指令中捕获信息，构建描述领域的一般性逻辑贝叶斯图。接着，我们生成逻辑骨架以指导语言模型训练，将领域知识注入语言模型。最后，我们交替优化图搜索策略与语言模型直至收敛。实验结果表明，与传统语言模型相比，Logical-GLM在仅使用较小规模训练数据和较少参数的情况下，仍能保持高效性和有效性。得益于内化的领域知识，我们的方法能够生成逻辑更正确的指令文本。此外，逻辑图的使用反映了语言模型的内在机制，从而提升了黑盒模型的可解释性。