Despite the great success of neural visual generative models in recent years, integrating them with strong symbolic knowledge reasoning systems remains a challenging task. The main challenges are two-fold: one is symbol assignment, i.e. bonding latent factors of neural visual generators with meaningful symbols from knowledge reasoning systems. Another is rule learning, i.e. learning new rules, which govern the generative process of the data, to augment the knowledge reasoning systems. To deal with these symbol grounding problems, we propose a neural-symbolic learning approach, Abductive Visual Generation (AbdGen), for integrating logic programming systems with neural visual generative models based on the abductive learning framework. To achieve reliable and efficient symbol assignment, the quantized abduction method is introduced for generating abduction proposals by the nearest-neighbor lookups within semantic codebooks. To achieve precise rule learning, the contrastive meta-abduction method is proposed to eliminate wrong rules with positive cases and avoid less-informative rules with negative cases simultaneously. Experimental results on various benchmark datasets show that compared to the baselines, AbdGen requires significantly fewer instance-level labeling information for symbol assignment. Furthermore, our approach can effectively learn underlying logical generative rules from data, which is out of the capability of existing approaches.

翻译：尽管近年来神经视觉生成模型取得了巨大成功，但将其与强符号知识推理系统集成仍是一项具有挑战性的任务。主要挑战有两个方面：其一是符号分配，即需要将神经视觉生成器的潜在因子与知识推理系统中的有意义的符号绑定；其二是规则学习，即需要学习支配数据生成过程的新规则，以增强知识推理系统。为了解决这些符号锚定问题，我们提出了一种神经符号学习方法——溯因视觉生成（AbdGen），该方法基于溯因学习框架将逻辑编程系统与神经视觉生成模型相结合。为了实现可靠且高效的符号分配，我们引入了量化溯因方法，通过语义码本中的最近邻查找生成溯因提议。为了实现精确的规则学习，我们提出了对比元溯因方法，该方法能够同时利用正例消除错误规则，并利用反例排除信息量不足的规则。在多个基准数据集上的实验结果表明，与基线方法相比，AbdGen在符号分配所需的实例级标注信息方面显著减少。此外，我们的方法能够有效地从数据中学习底层的逻辑生成规则，这是现有方法所不具备的能力。