Continual Learning trains models on a stream of data, with the aim of learning new information without forgetting previous knowledge. Given the dynamic nature of such environments, explaining the predictions of these models can be challenging. We study the behavior of SHAP values explanations in Continual Learning and propose an evaluation protocol to robustly assess the change of explanations in Class-Incremental scenarios. We observed that, while Replay strategies enforce the stability of SHAP values in feedforward/convolutional models, they are not able to do the same with fully-trained recurrent models. We show that alternative recurrent approaches, like randomized recurrent models, are more effective in keeping the explanations stable over time.

翻译：持续学习通过在数据流上训练模型，旨在学习新信息的同时不遗忘先前知识。鉴于此类环境的动态特性，对这些模型预测的解释可能具有挑战性。我们研究了持续学习中SHAP值解释的行为，并提出了一种评估协议，以稳健地评估类增量场景下解释的变化。我们观察到，虽然重放策略能强制前馈/卷积模型中SHAP值的稳定性，但无法对完全训练的循环模型实现同样的效果。我们表明，替代性循环方法（如随机循环模型）在随时间保持解释稳定性方面更为有效。