Current learning models often struggle with human-like systematic generalization, particularly in learning compositional rules from limited data and extrapolating them to novel combinations. We introduce the Neural-Symbolic Recursive Machine (NSR), whose core is a Grounded Symbol System (GSS), allowing for the emergence of combinatorial syntax and semantics directly from training data. The NSR employs a modular design that integrates neural perception, syntactic parsing, and semantic reasoning. These components are synergistically trained through a novel deduction-abduction algorithm. Our findings demonstrate that NSR's design, imbued with the inductive biases of equivariance and compositionality, grants it the expressiveness to adeptly handle diverse sequence-to-sequence tasks and achieve unparalleled systematic generalization. We evaluate NSR's efficacy across four challenging benchmarks designed to probe systematic generalization capabilities: SCAN for semantic parsing, PCFG for string manipulation, HINT for arithmetic reasoning, and a compositional machine translation task. The results affirm NSR's superiority over contemporary neural and hybrid models in terms of generalization and transferability.
翻译:当前学习模型常难以实现类人的系统性泛化,尤其在从有限数据中学习组合规则并将其外推至新组合方面。我们提出神经符号递归机器(NSR),其核心是基础符号系统(GSS),该机制使组合句法和语义能直接从训练数据中涌现。NSR采用模块化设计,集成神经感知、句法解析和语义推理三大组件,通过新型演绎-溯因算法协同训练。研究表明,NSR的设计蕴含等变性和组合性的归纳偏置,使其具备充分表达能力,能灵活处理多种序列到序列任务,并实现无与伦比的系统性泛化。我们在四项旨在探测系统性泛化能力的挑战性基准上评估NSR:语义解析任务SCAN、字符串操作任务PCFG、算术推理任务HINT以及组合型机器翻译任务。实验结果证实,NSR在泛化性与迁移性方面显著优于当前神经模型与混合模型。