Language models (LMs) proficiency in handling deterministic symbolic reasoning and rule-based tasks remains limited due to their dependency implicit learning on textual data. To enable fully rule comprehension ability, we explore how to incorporate compiled neural networks (CoNNs) which weight is specially designed into the architecture of LMs, to achieve high accuracy and robust performance. CoNNs are transformer-based neural networks that execute rules through artificially generated attention weights. Our method, which call "Neural Comprehension", by incorporating CoNN modules into the LM, the framework effectively tackles rule-intensive challenges. Our experiments on symbolic reasoning tasks and real-world arithmetic reasoning tasks demonstrate the superior performance of our method compared to existing techniques. Furthermore, our LM achieves flawless execution on symbolic operations tasks, highlighting the potential of our method in enabling LMs to possess true symbolic comprehension capabilities. Our code is publicly available at: https://github.com/WENGSYX/Neural-Comprehension.

翻译：语言模型在处理确定性符号推理和基于规则的任务时能力有限，这源于其对文本数据的隐性学习依赖。为实现完全的规则理解能力，我们探索如何将具有专门设计的权重的编译神经网络（CoNNs）融入语言模型架构，以实现高精度和稳健性能。CoNNs是基于Transformer的神经网络，通过人工生成的注意力权重执行规则。我们的方法称为“神经理解”（Neural Comprehension），通过将CoNN模块整合到语言模型中，该框架能有效应对规则密集型挑战。我们在符号推理任务和真实算术推理任务上的实验表明，与现有技术相比，我们的方法具有优越性能。此外，我们的语言模型在符号运算任务上实现了完美执行，凸显了该方法在赋予语言模型真正符号理解能力方面的潜力。我们的代码公开于：https://github.com/WENGSYX/Neural-Comprehension。