Continual Learning trains models on a stream of data, with the aim of learning new information without forgetting previous knowledge. Given the dynamic nature of such environments, explaining the predictions of these models can be challenging. We study the behavior of SHAP values explanations in Continual Learning and propose an evaluation protocol to robustly assess the change of explanations in Class-Incremental scenarios. We observed that, while Replay strategies enforce the stability of SHAP values in feedforward/convolutional models, they are not able to do the same with fully-trained recurrent models. We show that alternative recurrent approaches, like randomized recurrent models, are more effective in keeping the explanations stable over time.

翻译：持续学习在数据流上训练模型，旨在学习新信息的同时不遗忘先前知识。鉴于此类环境的动态特性，解释这些模型的预测可能具有挑战性。我们研究了持续学习中SHAP值解释的行为，并提出了一种评估协议，以稳健地评估类别增量场景中解释的变化。我们观察到，虽然回放策略在前馈/卷积模型中强制了SHAP值的稳定性，但在完全训练好的循环模型中却无法实现相同效果。我们表明，替代性循环方法（如随机化循环模型）在长期保持解释稳定性方面更为有效。