Deep Learning (DL) techniques have achieved remarkable successes in recent years. However, their ability to generalize and execute reasoning tasks remains a challenge. A potential solution to this issue is Neuro-Symbolic Integration (NeSy), where neural approaches are combined with symbolic reasoning. Most of these methods exploit a neural network to map perceptions to symbols and a logical reasoner to predict the output of the downstream task. These methods exhibit superior generalization capacity compared to fully neural architectures. However, they suffer from several issues, including slow convergence, learning difficulties with complex perception tasks, and convergence to local minima. This paper proposes a simple yet effective method to ameliorate these problems. The key idea involves pretraining a neural model on the downstream task. Then, a NeSy model is trained on the same task via transfer learning, where the weights of the perceptual part are injected from the pretrained network. The key observation of our work is that the neural network fails to generalize only at the level of the symbolic part while being perfectly capable of learning the mapping from perceptions to symbols. We have tested our training strategy on various SOTA NeSy methods and datasets, demonstrating consistent improvements in the aforementioned problems.

翻译：深度学习技术近年来取得了显著成功。然而，其泛化能力和执行推理任务的能力仍然面临挑战。神经符号集成是解决这一问题的潜在方案，该方法将神经方法与符号推理相结合。大多数此类方法利用神经网络将感知映射到符号，并借助逻辑推理器预测下游任务的输出。与完全神经架构相比，这些方法展现出更优越的泛化能力。然而，它们也存在若干问题，包括收敛速度慢、复杂感知任务学习困难以及易陷入局部极小值。本文提出一种简单而有效的方法来改善这些问题。其核心思想是在下游任务上预训练神经模型，随后通过迁移学习在相同任务上训练神经符号模型——其中感知部分的权重从预训练网络注入。我们工作的关键发现是：神经网络仅在符号层面存在泛化失败，而完全能够学习从感知到符号的映射。我们在多种先进神经符号方法与数据集上验证了所提出的训练策略，结果表明该方法能持续改善上述问题。