Continual learning has emerged as an increasingly important challenge across various tasks, including Spoken Language Understanding (SLU). In SLU, its objective is to effectively handle the emergence of new concepts and evolving environments. The evaluation of continual learning algorithms typically involves assessing the model's stability, plasticity, and generalizability as fundamental aspects of standards. However, existing continual learning metrics primarily focus on only one or two of the properties. They neglect the overall performance across all tasks, and do not adequately disentangle the plasticity versus stability/generalizability trade-offs within the model. In this work, we propose an evaluation methodology that provides a unified evaluation on stability, plasticity, and generalizability in continual learning. By employing the proposed metric, we demonstrate how introducing various knowledge distillations can improve different aspects of these three properties of the SLU model. We further show that our proposed metric is more sensitive in capturing the impact of task ordering in continual learning, making it better suited for practical use-case scenarios.

翻译：持续学习已成为涵盖口语理解（SLU）在内的多项任务中日益重要的挑战。在SLU中，其目标在于有效处理新概念的出现和环境演变。持续学习算法的评估通常涉及对模型稳定性、可塑性和泛化性这三项基本标准的考察。然而，现有的持续学习指标主要仅关注其中一项或两项性质，不仅忽略了所有任务上的整体性能，也未能充分解耦模型内部可塑性与稳定性/泛化性之间的权衡。本文提出了一种评估方法，可为持续学习中的稳定性、可塑性和泛化性提供统一评估。通过采用所提出的指标，我们展示了引入不同知识蒸馏方式如何提升SLU模型这三方面性质的不同维度。进一步研究表明，我们提出的指标在捕捉持续学习中任务顺序的影响方面更为敏锐，因此更适用于实际应用场景。