Compositional Natural Language Inference has been explored to assess the true abilities of neural models to perform NLI. Yet, current evaluations assume models to have full access to all primitive inferences in advance, in contrast to humans that continuously acquire inference knowledge. In this paper, we introduce the Continual Compositional Generalization in Inference (C2Gen NLI) challenge, where a model continuously acquires knowledge of constituting primitive inference tasks as a basis for compositional inferences. We explore how continual learning affects compositional generalization in NLI, by designing a continual learning setup for compositional NLI inference tasks. Our experiments demonstrate that models fail to compositionally generalize in a continual scenario. To address this problem, we first benchmark various continual learning algorithms and verify their efficacy. We then further analyze C2Gen, focusing on how to order primitives and compositional inference types and examining correlations between subtasks. Our analyses show that by learning subtasks continuously while observing their dependencies and increasing degrees of difficulty, continual learning can enhance composition generalization ability.

翻译：组合自然语言推理已被用于评估神经模型执行NLI的真实能力。然而，当前的评估假设模型能预先完全访问所有原始推理知识，这与人类持续获取推理知识的方式形成对比。本文提出了推理中的持续组合泛化挑战，要求模型持续获取构成性原始推理任务的知识，以此作为组合推理的基础。通过为组合NLI推理任务设计持续学习框架，我们探究了持续学习如何影响NLI中的组合泛化能力。实验表明，模型在持续学习场景中难以实现组合泛化。针对该问题，我们首先对多种持续学习算法进行基准测试并验证其有效性。随后进一步分析C2Gen框架，重点关注原始推理与组合推理类型的排序策略，并检验子任务间的相关性。分析表明，通过持续学习子任务并观察其依赖关系与难度递增规律，持续学习能够有效提升组合泛化能力。