Numerous neuro-symbolic approaches have recently been proposed typically with the goal of adding symbolic knowledge to the output layer of a neural network. Ideally, such losses maximize the probability that the neural network's predictions satisfy the underlying domain. Unfortunately, this type of probabilistic inference is often computationally infeasible. Neuro-symbolic approaches therefore commonly resort to fuzzy approximations of this probabilistic objective, sacrificing sound probabilistic semantics, or to sampling which is very seldom feasible. We approach the problem by first assuming the constraint decomposes conditioned on the features learned by the network. We iteratively strengthen our approximation, restoring the dependence between the constraints most responsible for degrading the quality of the approximation. This corresponds to computing the mutual information between pairs of constraints conditioned on the network's learned features, and may be construed as a measure of how well aligned the gradients of two distributions are. We show how to compute this efficiently for tractable circuits. We test our approach on three tasks: predicting a minimum-cost path in Warcraft, predicting a minimum-cost perfect matching, and solving Sudoku puzzles, observing that it improves upon the baselines while sidestepping intractability.

翻译：近期提出了大量神经符号方法，其典型目标是将符号知识添加到神经网络的输出层。理想情况下，此类损失函数可最大化神经网络预测结果满足底层域的概率。遗憾的是，这类概率推断通常因计算难以实现而受阻。因此，神经符号方法通常采用对该概率目标的模糊近似——牺牲合理的概率语义，或采用极少可行的采样方法。我们通过假设约束条件在给定网络所学特征时是可分解的来处理该问题。我们迭代式地强化近似过程，恢复对近似质量影响最大的约束条件之间的依赖关系。这相当于计算网络所学特征条件下约束对之间的互信息，可被理解为两个分布梯度对齐程度的度量。我们展示了如何针对可处理电路高效计算该量。我们在三个任务上测试了方法：预测《魔兽争霸》中的最小成本路径、预测最小成本完美匹配以及解决数独谜题，观察发现本方法在规避不可解性的同时提升了基线性能。