Deep Neural Networks (DNNs) based semantic segmentation of the robotic instruments and tissues can enhance the precision of surgical activities in robot-assisted surgery. However, in biological learning, DNNs cannot learn incremental tasks over time and exhibit catastrophic forgetting, which refers to the sharp decline in performance on previously learned tasks after learning a new one. Specifically, when data scarcity is the issue, the model shows a rapid drop in performance on previously learned instruments after learning new data with new instruments. The problem becomes worse when it limits releasing the dataset of the old instruments for the old model due to privacy concerns and the unavailability of the data for the new or updated version of the instruments for the continual learning model. For this purpose, we develop a privacy-preserving synthetic continual semantic segmentation framework by blending and harmonizing (i) open-source old instruments foreground to the synthesized background without revealing real patient data in public and (ii) new instruments foreground to extensively augmented real background. To boost the balanced logit distillation from the old model to the continual learning model, we design overlapping class-aware temperature normalization (CAT) by controlling model learning utility. We also introduce multi-scale shifted-feature distillation (SD) to maintain long and short-range spatial relationships among the semantic objects where conventional short-range spatial features with limited information reduce the power of feature distillation. We demonstrate the effectiveness of our framework on the EndoVis 2017 and 2018 instrument segmentation dataset with a generalized continual learning setting. Code is available at~\url{https://github.com/XuMengyaAmy/Synthetic_CAT_SD}.

翻译：基于深度神经网络（DNN）的机器人手术器械与组织语义分割可提升机器人辅助手术中操作活动的精度。然而在生物学习过程中，DNN 无法随时间增量式学习新任务，并会出现灾难性遗忘——即学习新任务后先前学到的任务性能急剧下降。具体而言，当面临数据稀缺问题时，模型在学习带有新器械的新数据后，其对先前学过的器械识别性能会快速衰减。当因隐私问题无法公开旧器械数据集以获取旧模型，且由于新型号或更新版本器械的数据不可得导致无法进行连续学习时，该问题将更加严重。为此，我们提出一种融合协调以下两种策略的隐私保护合成式连续语义分割框架：（i）在不泄露真实患者数据的前提下，将开源旧器械前景合成至虚拟背景中；（ii）将新器械前景合成至经广泛增强的真实背景中。为提升从旧模型到连续学习模型的平衡对数蒸馏效果，我们设计了重叠感知的类别级温度归一化（CAT）方法，通过控制模型学习效用实现优化。同时引入多尺度偏移特征蒸馏（SD）技术，以保持语义对象间的长程与短程空间关系——传统短程空间特征因信息有限而削弱了特征蒸馏能力。我们在 EndoVis 2017 和 2018 器械分割数据集上验证了该框架在泛化连续学习设定下的有效性。代码已开源：\url{https://github.com/XuMengyaAmy/Synthetic_CAT_SD}