Detecting negatives (such as non-entailment relationships, unanswerable questions, and false claims) is an important and challenging aspect of many natural language understanding tasks. Though manually collecting challenging negative examples can help models detect them, it is both costly and domain-specific. In this work, we propose Self-labeled Counterfactuals for Extrapolating to Negative Examples (SCENE), an automatic method for synthesizing training data that greatly improves models' ability to detect challenging negative examples. In contrast with standard data augmentation, which synthesizes new examples for existing labels, SCENE can synthesize negative examples zero-shot from only positive ones. Given a positive example, SCENE perturbs it with a mask infilling model, then determines whether the resulting example is negative based on a self-training heuristic. With access to only answerable training examples, SCENE can close 69.6% of the performance gap on SQuAD 2.0, a dataset where half of the evaluation examples are unanswerable, compared to a model trained on SQuAD 2.0. Our method also extends to boolean question answering and recognizing textual entailment, and improves generalization from SQuAD to ACE-whQA, an out-of-domain extractive QA benchmark.

翻译：检测负例（如非蕴含关系、不可回答问题、虚假声明）是许多自然语言理解任务中的重要且富有挑战性的方面。尽管手动收集具有挑战性的负例有助于模型检测它们，但这种方法既成本高昂又局限于特定领域。本文提出一种用于外推至负例的自标注反事实方法（SCENE），该方法能自动合成训练数据，显著提升模型检测复杂负例的能力。与标准数据增强（为现有标签合成新样例）不同，SCENE 仅需正例即可实现零样本负例合成。给定一个正例，SCENE 通过掩码填充模型对其扰动，随后基于自训练启发式规则判断生成的样例是否为负例。仅利用可回答的训练样例，SCENE 即可在 SQuAD 2.0（该数据集中半数评估样例为不可回答问题）上缩小 69.6% 的性能差距，其效果接近直接使用 SQuAD 2.0 训练的模型。此外，本方法可扩展至布尔问答和文本蕴含识别任务，并提升从 SQuAD 到域外抽取式问答基准 ACE-whQA 的泛化能力。