Compositional Natural Language Inference has been explored to assess the true abilities of neural models to perform NLI. Yet, current evaluations assume models to have full access to all primitive inferences in advance, in contrast to humans that continuously acquire inference knowledge. In this paper, we introduce the Continual Compositional Generalization in Inference (C2Gen NLI) challenge, where a model continuously acquires knowledge of constituting primitive inference tasks as a basis for compositional inferences. We explore how continual learning affects compositional generalization in NLI, by designing a continual learning setup for compositional NLI inference tasks. Our experiments demonstrate that models fail to compositionally generalize in a continual scenario. To address this problem, we first benchmark various continual learning algorithms and verify their efficacy. We then further analyze C2Gen, focusing on how to order primitives and compositional inference types and examining correlations between subtasks. Our analyses show that by learning subtasks continuously while observing their dependencies and increasing degrees of difficulty, continual learning can enhance composition generalization ability.

翻译：组合自然语言推理已被用于评估神经模型执行NLI的真正能力。然而，当前评估假设模型能提前完全访问所有原始推理，这与人类持续获取推理知识的方式形成对比。本文提出了推理中的持续组合泛化挑战（C2Gen NLI），要求模型持续获取构成原始推理任务的知识，作为组合推理的基础。我们通过设计组合NLI推理任务的持续学习设置，探究持续学习如何影响NLI中的组合泛化。实验表明，模型在持续场景中无法实现组合泛化。为解决此问题，我们首先对多种持续学习算法进行基准测试并验证其有效性。随后进一步分析C2Gen，重点关注原始推理与组合推理类型的顺序安排，并考察子任务间的相关性。分析表明，通过在持续学习中考虑子任务依赖关系并逐步增加难度，持续学习能够增强组合泛化能力。