In current Large Language Models we can trust the production of smoothly flowing prose on the basis of the principles of machine learning. However, there is no comparably principled basis to justify trust in the content of the text produced. It appears to be conventional wisdom that addressing this issue by adding more principled reasoning is not computationally affordable. Here we propose a principled method of reasoning that is efficient enough to be practical for large language models. Further, the method allows the retention of much of the currently used software and hardware base. Our method for improving the functioning of large language models consists of a first stage of preprocessing that recodes the data to a Unary Relational Integracode that is more explicit about the relationships among the objects described in the text, followed as a second stage by a standard but possibly streamlined machine learning process that then also learns to predict these relationships. The method may be viewed as realizing a world model and applying beyond natural language, to vision and actions, for example, where the multiple properties of an object referred to in an input are brought together explicitly, rather than remaining distributed in the various references to it in the input. We articulate its advantages in terms of Robust Logic, a system for performing principled chaining on learned, and hence uncertain, information. We show that this recoding has the surprising and fortuitous property that, while succinct, it makes the task of learning a core subset of relational rules that hold in the world described in the training data polynomial time learnable in a defined sense, the polynomial depending on the complexity of the rule. This gives support for sound reasoning within each single call of the learned classifier as well as between multiple calls.

翻译：在当前的大规模语言模型中，我们可以基于机器学习原理信任其生成流畅散文的能力。然而，对于所生成文本内容的可信度，却缺乏同样原则性的依据。普遍观点认为，通过增加更具原则性的推理来解决这一问题在计算上不可行。本文提出了一种原则性推理方法，其效率足以在大规模语言模型中实际应用。此外，该方法允许保留目前使用的大部分软件和硬件基础。我们改进大规模语言模型功能的方法包括两个阶段：第一阶段进行预处理，将数据重新编码为一种更明确表达文本中描述对象之间关系的“一元关系集成码”；第二阶段采用标准但可能精简的机器学习过程，该过程同时学习预测这些关系。该方法可被视为实现了一种世界模型，并应用于自然语言之外的领域，例如视觉和动作，其中输入中所指对象的多个属性被显式整合，而非分散在输入中的各个引用中。我们通过“鲁棒逻辑”系统阐述了其优势，该系统能对已学习因而具有不确定性的信息进行原则性链式推理。我们证明，这种重新编码具有一个令人惊讶且偶然的特性：虽然简洁，但它使得学习训练数据中描述世界里存在的关系规则核心子集的任务在特定意义上是多项式时间可学习的，该多项式取决于规则的复杂度。这为单次调用学习分类器内以及多次调用之间的合理推理提供了支持。