Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations (or ``chain-of-thought'' (CoT)) for in-context learning. On the other hand, these reasoning tasks are usually presumed to be more approachable for symbolic programming. To make progress towards understanding in-context learning, we curate synthetic datasets containing equivalent (natural, symbolic) data pairs, where symbolic examples contain first-order logic rules and predicates from knowledge bases (KBs). Then we revisit neuro-symbolic approaches and use Language Models as Logic Programmer (LMLP) that learns from demonstrations containing logic rules and corresponding examples to iteratively reason over KBs, recovering Prolog's backward chaining algorithm. Comprehensive experiments are included to systematically compare LMLP with CoT in deductive reasoning settings, showing that LMLP enjoys more than 25% higher accuracy than CoT on length generalization benchmarks even with fewer parameters.

翻译：受过培训的语言模型(LMS)已经展示了使用解释(或“思维链”)进行内流学习的非凡推理性能。另一方面,这些推理任务通常被推定为更适合象征性的编程。为了在理解内流学习方面取得进展,我们翻译了包含等同(自然的、象征性的)数据对的合成数据集,其中象征性的例子包含一阶逻辑规则和知识基础(KBs)的前提。然后,我们重新审视神经-顺流方法,并使用语言模型作为逻辑程序员(LMLP),这些模型从包含逻辑规则的演示中学习,并用相应实例来了解相对于KBs(恢复Prolog的后向链算法)的迭代原因。我们进行了全面实验,系统地将LMLP与COT在推理设置中进行比较,表明LMLP在一般化基准方面比CT高出25%以上,即使参数更少。