Robust semantic segmentation of intraoperative image data could pave the way for automatic surgical scene understanding and autonomous robotic surgery. Geometric domain shifts, however, although common in real-world open surgeries due to variations in surgical procedures or situs occlusions, remain a topic largely unaddressed in the field. To address this gap in the literature, we (1) present the first analysis of state-of-the-art (SOA) semantic segmentation networks in the presence of geometric out-of-distribution (OOD) data, and (2) address generalizability with a dedicated augmentation technique termed "Organ Transplantation" that we adapted from the general computer vision community. According to a comprehensive validation on six different OOD data sets comprising 600 RGB and hyperspectral imaging (HSI) cubes from 33 pigs semantically annotated with 19 classes, we demonstrate a large performance drop of SOA organ segmentation networks applied to geometric OOD data. Surprisingly, this holds true not only for conventional RGB data (drop of Dice similarity coefficient (DSC) by 46 %) but also for HSI data (drop by 45 %), despite the latter's rich information content per pixel. Using our augmentation scheme improves on the SOA DSC by up to 67 % (RGB) and 90 % (HSI) and renders performance on par with in-distribution performance on real OOD test data. The simplicity and effectiveness of our augmentation scheme makes it a valuable network-independent tool for addressing geometric domain shifts in semantic scene segmentation of intraoperative data. Our code and pre-trained models will be made publicly available.

翻译：术中图像数据的鲁棒语义分割可为自动手术场景理解和自主机器人手术铺平道路。然而，几何域偏移（由于手术操作差异或术野遮挡在真实开放手术中常见）仍是一个在领域内未得到充分关注的问题。为填补文献中的这一空白，我们：（1）首次分析了现有最优（SOA）语义分割网络在几何分布外（OOD）数据上的表现；（2）通过一种名为“器官移植”（Organ Transplantation）的专用数据增强技术（改编自通用计算机视觉领域）解决了泛化性问题。基于对33头猪的600个RGB和高光谱成像（HSI）数据立方体（包含19类语义标注）在六种不同OOD数据集上的综合验证，我们发现SOA器官分割网络应用于几何OOD数据时存在显著性能下降。令人惊讶的是，这一现象不仅适用于传统RGB数据（Dice相似系数（DSC）下降46%），也适用于每像素信息更丰富的HSI数据（DSC下降45%）。采用我们的增强方案后，SOA的DSC在RGB和HSI数据上分别提升高达67%和90%，使其在真实OOD测试数据上的表现与分布内性能相当。该增强方案简洁高效，成为处理术中数据语义场景分割中几何域偏移的通用网络工具。我们的代码和预训练模型将公开发布。