Neuro-symbolic neural networks have been extensively studied to integrate symbolic operations with neural networks, thereby improving systematic generalization. Specifically, Tensor Product Representation (TPR) framework enables neural networks to perform differentiable symbolic operations by encoding the symbolic structure of data within vector spaces. However, TPR-based neural networks often struggle to decompose unseen data into structured TPR representations, undermining their symbolic operations. To address this decomposition problem, we propose a Discrete Dictionary-based Decomposition (D3) layer designed to enhance the decomposition capabilities of TPR-based models. D3 employs discrete, learnable key-value dictionaries trained to capture symbolic features essential for decomposition operations. It leverages the prior knowledge acquired during training to generate structured TPR representations by mapping input data to pre-learned symbolic features within these dictionaries. D3 is a straightforward drop-in layer that can be seamlessly integrated into any TPR-based model without modifications. Our experimental results demonstrate that D3 significantly improves the systematic generalization of various TPR-based models while requiring fewer additional parameters. Notably, D3 outperforms baseline models on the synthetic task that demands the systematic decomposition of unseen combinatorial data.

翻译：神经符号神经网络已被广泛研究，旨在将符号操作与神经网络相结合，从而提升系统泛化能力。具体而言，张量积表示框架通过将数据的符号结构编码到向量空间中，使神经网络能够执行可微分的符号操作。然而，基于TPR的神经网络往往难以将未见数据分解为结构化的TPR表示，这削弱了其符号操作能力。为解决这一分解问题，我们提出了一种基于离散词典的分解层，旨在增强基于TPR模型的分解能力。D3层采用离散、可学习的键值词典进行训练，以捕获分解操作所必需的符号特征。它利用训练过程中获得的先验知识，通过将输入数据映射到词典中预先学习到的符号特征，生成结构化的TPR表示。D3层是一种简单的即插即用层，无需修改即可无缝集成到任何基于TPR的模型中。我们的实验结果表明，D3层在仅需少量额外参数的情况下，显著提升了多种基于TPR模型的系统泛化性能。值得注意的是，在需要对未见组合数据进行系统分解的合成任务上，D3层的表现优于基线模型。