Natural Language Inference (NLI) is the task of inferring whether the hypothesis can be justified by the given premise. Basically, we classify the hypothesis into three labels(entailment, neutrality and contradiction) given the premise. NLI was well studied by the previous researchers. A number of models, especially the transformer based ones, have achieved significant improvement on these tasks. However, it is reported that these models are suffering when they are dealing with hard datasets. Particularly, they perform much worse when dealing with unseen out-of-distribution premise and hypothesis. They may not understand the semantic content but learn the spurious correlations. In this work, we propose the data augmentation and preprocessing methods to solve the word overlap, numerical reasoning and length mismatch problems. These methods are general methods that do not rely on the distribution of the testing data and they help improve the robustness of the models.
翻译:自然语言推理(NLI)的任务是根据给定前提推断假设是否成立。本质上,我们是在给定前提的情况下将假设分类为三种标签(蕴含、中性和矛盾)。先前的研究者对NLI进行了深入探讨,许多模型,特别是基于Transformer的模型,在这些任务上取得了显著进展。然而,据报道,这些模型在处理困难数据集时表现不佳。尤其是在处理未见过的、分布外的前提和假设时,其性能明显下降。这些模型可能并未真正理解语义内容,而是学习了虚假的统计关联。在本研究中,我们提出了数据增强与预处理方法,以解决词汇重叠、数值推理和长度不匹配等问题。这些方法是通用方法,不依赖于测试数据的分布,有助于提升模型的鲁棒性。