To assist surgeons in the operating theatre, surgical phase recognition is critical for developing computer-assisted surgical systems, which requires comprehensive understanding of surgical videos. Although existing studies made great progress, there are still two significant limitations worthy of improvement. First, due to the compromise of resource consumption, frame-wise visual features are extracted by 2D networks and disregard spatial and temporal knowledge of surgical actions, which hinders subsequent inter-frame modeling for phase prediction. Second, these works simply utilize ordinary classification loss with one-hot phase labels to optimize the phase predictions, and cannot fully explore surgical videos under inadequate supervision. To overcome these two limitations, we propose a Surgical Temporal Action-aware Network with sequence Regularization, named STAR-Net, to recognize surgical phases more accurately from input videos. Specifically, we propose an efficient multi-scale surgical temporal action (MS-STA) module, which integrates visual features with spatial and temporal knowledge of surgical actions at the cost of 2D networks. Moreover, we devise the dual-classifier sequence regularization (DSR) to facilitate the training of STAR-Net by the sequence guidance of an auxiliary classifier with a smaller capacity. Our STAR-Net with MS-STA and DSR can exploit visual features of surgical actions with effective regularization, thereby leading to the superior performance of surgical phase recognition. Extensive experiments on a large-scale gastrectomy surgery dataset and the public Cholec80 benchmark prove that our STAR-Net significantly outperforms state-of-the-arts of surgical phase recognition.

翻译：为协助手术室中的外科医生，手术阶段识别对于开发计算机辅助手术系统至关重要，这需要对手术视频进行全面理解。尽管现有研究取得了重大进展，但仍存在两个值得改进的显著局限。首先，由于资源消耗的权衡，帧级视觉特征由2D网络提取，忽视了手术动作的时空知识，这阻碍了后续用于阶段预测的帧间建模。其次，这些工作仅利用带有独热阶段标签的普通分类损失来优化阶段预测，无法在监督不足的情况下充分探索手术视频。为克服这两个局限，我们提出了一种具有序列正则化的手术时序动作感知网络，命名为STAR-Net，以更准确地从输入视频中识别手术阶段。具体而言，我们提出了一种高效的多尺度手术时序动作（MS-STA）模块，该模块以2D网络的代价将视觉特征与手术动作的时空知识相结合。此外，我们设计了双分类器序列正则化（DSR），通过容量较小的辅助分类器的序列引导来促进STAR-Net的训练。我们的STAR-Net结合MS-STA和DSR，能够利用有效正则化挖掘手术动作的视觉特征，从而在手术阶段识别中取得优越性能。在大规模胃切除术手术数据集和公开的Cholec80基准上的大量实验证明，我们的STAR-Net显著优于最先进的手术阶段识别方法。