Modern surgical systems increasingly rely on intelligent scene understanding to provide timely situational awareness for enhanced intra-operative safety. Within this pipeline, surgical scene segmentation plays a central role in accurately perceiving operative events. Although recent deep learning models, particularly large-scale foundation models, achieve remarkable segmentation accuracy, their substantial computational demands and power consumption hinder real-time deployment in resource-constrained surgical environments. To address this limitation, we explore the emerging SNN as a promising paradigm for highly efficient surgical intelligence. However, their performance is still constrained by the scarcity of labeled surgical data and the inherently sparse nature of surgical video representations. To this end, we propose \textit{SpikeSurgSeg}, the first spike-driven video Transformer framework tailored for surgical scene segmentation with real-time potential on non-GPU platforms. To address the limited availability of surgical annotations, we introduce a surgical-scene masked autoencoding pretraining strategy for SNNs that enables robust spatiotemporal representation learning via layer-wise tube masking. Building on this pretrained backbone, we further adopt a lightweight spike-driven segmentation head that produces temporally consistent predictions while preserving the low-latency characteristics of SNNs. Extensive experiments on EndoVis18 and our in-house SurgBleed dataset demonstrate that SpikeSurgSeg achieves mIoU comparable to SOTA ANN-based models while reducing inference latency by at least $8\times$. Notably, it delivers over $20\times$ acceleration relative to most foundation-model baselines, underscoring its potential for time-critical surgical scene segmentation.

翻译：现代手术系统日益依赖智能场景理解，以提供及时的态势感知，从而增强术中安全性。在此流程中，手术场景分割在准确感知手术事件方面起着核心作用。尽管近期的深度学习模型，特别是大规模基础模型，实现了显著的分割精度，但其巨大的计算需求和功耗阻碍了在资源受限的手术环境中进行实时部署。为解决这一限制，我们探索了新兴的SNN作为一种高效手术智能的有前景范式。然而，其性能仍受限于标记手术数据的稀缺性以及手术视频表示固有的稀疏性。为此，我们提出了\textit{SpikeSurgSeg}，这是首个为手术场景分割量身定制的脉冲驱动视频Transformer框架，具备在非GPU平台上的实时潜能。针对手术标注数据有限的问题，我们引入了一种面向SNN的手术场景掩码自编码预训练策略，该策略通过分层管状掩码实现鲁棒的时空表示学习。基于此预训练骨干网络，我们进一步采用了一个轻量级的脉冲驱动分割头，该分割头在保持SNN低延迟特性的同时，能产生时间一致的预测。在EndoVis18和我们内部的SurgBleed数据集上进行的大量实验表明，SpikeSurgSeg实现了与基于ANN的SOTA模型相当的mIoU，同时将推理延迟降低了至少$8\times$。值得注意的是，相对于大多数基础模型基线，它提供了超过$20\times$的加速，突显了其在时间关键型手术场景分割中的潜力。