Generics express generalizations about the world (e.g., birds can fly) that are not universally true (e.g., newborn birds and penguins cannot fly). Commonsense knowledge bases, used extensively in NLP, encode some generic knowledge but rarely enumerate such exceptions and knowing when a generic statement holds or does not hold true is crucial for developing a comprehensive understanding of generics. We present a novel framework informed by linguistic theory to generate exemplars -- specific cases when a generic holds true or false. We generate ~19k exemplars for ~650 generics and show that our framework outperforms a strong GPT-3 baseline by 12.8 precision points. Our analysis highlights the importance of linguistic theory-based controllability for generating exemplars, the insufficiency of knowledge bases as a source of exemplars, and the challenges exemplars pose for the task of natural language inference.
翻译:泛称表达关于世界的普遍性概括(例如“鸟会飞”),而这些概括并非普遍成立(例如新生鸟和企鹅不会飞)。在自然语言处理中广泛使用的常识知识库编码了一些泛称知识,但很少列举此类例外情况,而了解泛称陈述何时成立或不成立对于全面理解泛称至关重要。我们提出了一种基于语言学理论的新框架,用于生成范例——即泛称成立或不成立的具体案例。我们为约650个泛称生成了约1.9万个范例,并表明我们的框架在精确率上比强大的GPT-3基线高出12.8个百分点。我们的分析凸显了基于语言学理论的可控性对生成范例的重要性、知识库作为范例来源的不足,以及范例对自然语言推理任务构成的挑战。