Weakly supervised surgical instrument segmentation with only instrument presence labels has been rarely explored in surgical domain. To mitigate the highly under-constrained challenges, we extend a two-stage weakly supervised segmentation paradigm with temporal attributes from two perspectives. From a temporal equivariance perspective, we propose a prototype-based temporal equivariance regulation loss to enhance pixel-wise consistency between adjacent features. From a semantic continuity perspective, we propose a class-aware temporal semantic continuity loss to constrain the semantic consistency between a global view of target frame and local non-discriminative regions of adjacent reference frame. To the best of our knowledge, WeakSurg is the first instrument-presence-only weakly supervised segmentation architecture to take temporal information into account for surgical scenarios. Extensive experiments are validated on Cholec80, an open benchmark for phase and instrument recognition. We annotate instance-wise instrument labels with fixed time-steps which are double checked by a clinician with 3-years experience. Our results show that WeakSurg compares favorably with state-of-the-art methods not only on semantic segmentation metrics but also on instance segmentation metrics.
翻译:弱监督手术器械分割(仅使用器械存在标签)在手术领域鲜有探索。为缓解高度欠约束的挑战,我们从两个角度扩展了具有时间属性的两阶段弱监督分割框架。从时间等变角度,我们提出基于原型的时间等变正则化损失,以增强相邻特征间的像素级一致性;从语义连续性角度,我们提出类别感知的时间语义连续性损失,用于约束目标帧全局视图与相邻参考帧局部非判别区域之间的语义一致性。据我们所知,WeakSurg是首个在手术场景中考虑时间信息的仅器械存在标签弱监督分割架构。我们在公开的器械与阶段识别基准数据集Cholec80上进行了大量实验验证,并以固定时间步长标注了实例级器械标签(经三年临床经验医师双重校验)。结果表明,WeakSurg不仅在语义分割指标上,更在实例分割指标上均优于现有最佳方法。