Despite the great success of neural visual generative models in recent years, integrating them with strong symbolic reasoning systems remains a challenging task. There are two levels of symbol grounding problems among the core challenges: the first is symbol assignment, i.e. mapping latent factors of neural visual generators to semantic-meaningful symbolic factors from the reasoning systems by learning from limited labeled data. The second is rule learning, i.e. learning new rules that govern the generative process to enhance the symbolic reasoning systems. To deal with these two problems, we propose a neurosymbolic learning approach, Abductive visual Generation (AbdGen), for integrating logic programming systems with neural visual generative models based on the abductive learning framework. To achieve reliable and efficient symbol grounding, the quantized abduction method is introduced for generating abduction proposals by the nearest-neighbor lookup within semantic codebooks. To achieve precise rule learning, the contrastive meta-abduction method is proposed to eliminate wrong rules with positive cases and avoid less informative rules with negative cases simultaneously. Experimental results show that compared to the baseline approaches, AbdGen requires significantly less labeled data for symbol assignment. Furthermore, AbdGen can effectively learn underlying logical generative rules from data, which is out of the capability of existing approaches. The code is released at this link: https://github.com/candytalking/AbdGen.

翻译：尽管近年来神经视觉生成模型取得了巨大成功，但将其与强大的符号推理系统整合仍是一项具有挑战性的任务。核心挑战中存在两个层次的符号基础问题：第一个是符号分配，即通过从有限标注数据中学习，将神经视觉生成器的潜在因子映射为推理系统中具有语义意义的符号因子；第二个是规则学习，即学习控制生成过程的新规则以增强符号推理系统。针对这两个问题，我们提出了一种神经符号学习方法——溯因视觉生成（AbdGen），基于溯因学习框架将逻辑编程系统与神经视觉生成模型相结合。为实现可靠高效的符号基础，我们引入了量化溯因方法，通过语义码本中的最近邻查找生成溯因提议；为实现精确规则学习，我们提出了对比元溯因方法，以同时消除错误规则（通过正例）和避免信息量不足的规则（通过反例）。实验结果表明，与基线方法相比，AbdGen在符号分配任务中所需的标注数据显著更少。此外，AbdGen还能有效从数据中学习潜在的逻辑生成规则，这是现有方法无法实现的能力。代码已发布在该链接：https://github.com/candytalking/AbdGen。