Prompt-based methods have gained increasing attention on NLP and shown validity on many downstream tasks. Many works have focused on mining these methods' potential for knowledge extraction, but few explore their ability to make logical reasoning. In this work, we focus on the effectiveness of the prompt-based methods on first-order logical reasoning and find that the bottleneck lies in logical negation. Based on our analysis, logical negation tends to result in spurious correlations to negative answers, while propositions without logical negation correlate to positive answers. To solve the problem, we propose a simple but effective method, Negation Augmenting and Negation Debiasing (NAND), which introduces negative propositions to prompt-based methods without updating parameters. Specifically, these negative propositions can counteract spurious correlations by providing "not" for all instances so that models cannot make decisions only by whether expressions contain a logical negation. Experiments on three datasets show that NAND not only solves the problem of calibrating logical negation but also significantly enhances prompt-based methods of logical reasoning without model retraining.
翻译:基于提示的方法在自然语言处理领域日益受到关注,并在许多下游任务中显示出有效性。许多研究致力于挖掘这些方法在知识提取方面的潜力,但鲜有探索其进行逻辑推理的能力。本研究聚焦于基于提示方法在一阶逻辑推理中的有效性,发现瓶颈在于逻辑否定。基于我们的分析,逻辑否定往往会导致与否定答案产生伪相关,而不含逻辑否定的命题则与肯定答案相关。为解决这一问题,我们提出一种简单而有效的方法——否定增强与否定去偏(NAND),该方法在不更新参数的情况下将否定命题引入基于提示的方法。具体而言,这些否定命题通过为所有实例提供"not"来抵消伪相关,使模型无法仅凭表达式中是否包含逻辑否定做出决策。在三个数据集上的实验表明,NAND不仅能解决逻辑否定的校准问题,还能在不重新训练模型的情况下显著增强基于提示方法的逻辑推理能力。