Reasoning over Commonsense Knowledge Bases (CSKB), i.e. CSKB reasoning, has been explored as a way to acquire new commonsense knowledge based on reference knowledge in the original CSKBs and external prior knowledge. Despite the advancement of Large Language Models (LLM) and prompt engineering techniques in various reasoning tasks, they still struggle to deal with CSKB reasoning. One of the problems is that it is hard for them to acquire explicit relational constraints in CSKBs from only in-context exemplars, due to a lack of symbolic reasoning capabilities (Bengio et al., 2021). To this end, we proposed **ConstraintChecker**, a plugin over prompting techniques to provide and check explicit constraints. When considering a new knowledge instance, ConstraintChecker employs a rule-based module to produce a list of constraints, then it uses a zero-shot learning module to check whether this knowledge instance satisfies all constraints. The acquired constraint-checking result is then aggregated with the output of the main prompting technique to produce the final output. Experimental results on CSKB Reasoning benchmarks demonstrate the effectiveness of our method by bringing consistent improvements over all prompting methods. Codes and data are available at \url{https://github.com/HKUST-KnowComp/ConstraintChecker}.
翻译:常识知识库(CSKB)推理旨在基于原始CSKB中的参考知识与外部先验知识获取新的常识知识。尽管大型语言模型(LLM)与提示工程技术在各种推理任务中取得进展,但其在CSKB推理任务中仍面临挑战。其中一个问题在于,由于缺乏符号推理能力(Bengio等人,2021),模型难以仅通过上下文示例获取CSKB中的显式关系约束。为此,我们提出**ConstraintChecker**——一种基于提示技术的插件,用于提供并验证显式约束。当处理新知识实例时,ConstraintChecker首先通过规则模块生成约束列表,继而利用零样本学习模块检查该实例是否满足所有约束。最终将约束验证结果与主提示技术的输出进行聚合,生成最终结果。在CSKB推理基准上的实验结果表明,该方法能在所有提示方法基础上带来持续改进,验证了其有效性。代码与数据已开源至 \url{https://github.com/HKUST-KnowComp/ConstraintChecker}。