The visual-question localized-answering (VQLA) system can serve as a knowledgeable assistant in surgical education. Except for providing text-based answers, the VQLA system can highlight the interested region for better surgical scene understanding. However, deep neural networks (DNNs) suffer from catastrophic forgetting when learning new knowledge. Specifically, when DNNs learn on incremental classes or tasks, their performance on old tasks drops dramatically. Furthermore, due to medical data privacy and licensing issues, it is often difficult to access old data when updating continual learning (CL) models. Therefore, we develop a non-exemplar continual surgical VQLA framework, to explore and balance the rigidity-plasticity trade-off of DNNs in a sequential learning paradigm. We revisit the distillation loss in CL tasks, and propose rigidity-plasticity-aware distillation (RP-Dist) and self-calibrated heterogeneous distillation (SH-Dist) to preserve the old knowledge. The weight aligning (WA) technique is also integrated to adjust the weight bias between old and new tasks. We further establish a CL framework on three public surgical datasets in the context of surgical settings that consist of overlapping classes between old and new surgical VQLA tasks. With extensive experiments, we demonstrate that our proposed method excellently reconciles learning and forgetting on the continual surgical VQLA over conventional CL methods. Our code is publicly accessible.

翻译：视觉问答定位（VQLA）系统可作为手术教育中的知识型助手。除提供基于文本的答案外，VQLA系统还能对感兴趣区域进行高亮显示，以增强对手术场景的理解。然而，深度神经网络在学习新知识时存在灾难性遗忘问题：当网络在增量类别或任务上学习时，其在旧任务上的性能会显著下降。此外，受医疗数据隐私及许可问题限制，更新持续学习模型时往往难以获取旧数据。为此，我们开发了一种无样例的持续手术VQLA框架，旨在探索并平衡深度神经网络在顺序学习范式中的刚性-可塑性权衡。我们重新审视了持续学习任务中的蒸馏损失，并提出刚性-可塑性感知蒸馏（RP-Dist）与自校准异构蒸馏（SH-Dist）以保留旧知识。同时引入权重对齐技术来调整新旧任务间的权重偏差。我们进一步在包含新旧手术VQLA任务重叠类别的三个公开手术数据集上构建了持续学习框架。大量实验表明，所提方法在持续手术VQLA任务中相较于传统持续学习方法更优地实现了学习与遗忘的平衡。我们的代码已公开。