Neural networks continue to struggle with compositional generalization, and this issue is exacerbated by a lack of massive pre-training. One successful approach for developing neural systems which exhibit human-like compositional generalization is \textit{hybrid} neurosymbolic techniques. However, these techniques run into the core issues that plague symbolic approaches to AI: scalability and flexibility. The reason for this failure is that at their core, hybrid neurosymbolic models perform symbolic computation and relegate the scalable and flexible neural computation to parameterizing a symbolic system. We investigate a \textit{unified} neurosymbolic system where transformations in the network can be interpreted simultaneously as both symbolic and neural computation. We extend a unified neurosymbolic architecture called the Differentiable Tree Machine in two central ways. First, we significantly increase the model's efficiency through the use of sparse vector representations of symbolic structures. Second, we enable its application beyond the restricted set of tree2tree problems to the more general class of seq2seq problems. The improved model retains its prior generalization capabilities and, since there is a fully neural path through the network, avoids the pitfalls of other neurosymbolic techniques that elevate symbolic computation over neural computation.

翻译：神经网络在组合泛化方面仍面临挑战，而大规模预训练的缺乏进一步加剧了这一问题。开发具有类人组合泛化能力的神经系统中，一种成功的方法是采用\textit{混合}神经符号技术。然而，这些技术仍难以规避困扰符号人工智能方法的核心问题：可扩展性与灵活性。其根本原因在于，混合神经符号模型本质上执行符号计算，而将可扩展且灵活的神经计算降级为符号系统的参数化过程。本研究探索一种\textit{统一}神经符号系统，其中网络变换可同时被解释为符号计算与神经计算。我们从两个核心维度拓展了名为可微分树机的统一神经符号架构：首先，通过采用符号结构的稀疏向量表示，显著提升了模型效率；其次，将其应用范围从受限的树到树问题类别扩展至更通用的序列到序列问题类别。改进后的模型保留了原有的泛化能力，且由于网络中存在完整的神经路径，避免了其他将符号计算凌驾于神经计算之上的神经符号技术的固有缺陷。