This paper introduces the ``SurgT: Surgical Tracking" challenge which was organised in conjunction with MICCAI 2022. There were two purposes for the creation of this challenge: (1) the establishment of the first standardised benchmark for the research community to assess soft-tissue trackers; and (2) to encourage the development of unsupervised deep learning methods, given the lack of annotated data in surgery. A dataset of 157 stereo endoscopic videos from 20 clinical cases, along with stereo camera calibration parameters, have been provided. Participants were assigned the task of developing algorithms to track the movement of soft tissues, represented by bounding boxes, in stereo endoscopic videos. At the end of the challenge, the developed methods were assessed on a previously hidden test subset. This assessment uses benchmarking metrics that were purposely developed for this challenge, to verify the efficacy of unsupervised deep learning algorithms in tracking soft-tissue. The metric used for ranking the methods was the Expected Average Overlap (EAO) score, which measures the average overlap between a tracker's and the ground truth bounding boxes. Coming first in the challenge was the deep learning submission by ICVS-2Ai with a superior EAO score of 0.617. This method employs ARFlow to estimate unsupervised dense optical flow from cropped images, using photometric and regularization losses. Second, Jmees with an EAO of 0.583, uses deep learning for surgical tool segmentation on top of a non-deep learning baseline method: CSRT. CSRT by itself scores a similar EAO of 0.563. The results from this challenge show that currently, non-deep learning methods are still competitive. The dataset and benchmarking tool created for this challenge have been made publicly available at https://surgt.grand-challenge.org/.
翻译:本文介绍了与MICCAI 2022联合举办的“SurgT:手术追踪”挑战赛。本次挑战赛旨在实现两个目标:(1)为研究社区建立首个标准化基准,以评估软组织追踪器;(2)鉴于手术领域标注数据的匮乏,鼓励开发无监督深度学习方法。挑战赛提供了来自20个临床案例的157个立体内窥镜视频数据集,并附有立体相机标定参数。参赛者的任务是开发算法,以跟踪由边界框表示的软组织在立体内窥镜视频中的运动。挑战赛结束时,在先前隐藏的测试子集上对开发的方法进行了评估。评估采用了专为本次挑战赛开发的基准测试指标,以验证无监督深度学习算法在追踪软组织方面的有效性。用于方法排序的指标是期望平均重叠(EAO)分数,该分数衡量追踪器与真实边界框之间的平均重叠程度。挑战赛第一名由ICVS-2Ai团队的深度学习方法获得,其EAO分数达到0.617。该方法采用ARFlow从裁剪图像中估计无监督密集光流,利用光度损失和正则化损失。第二名Jmees团队的EAO分数为0.583,其在非深度学习基线方法CSRT之上,使用深度学习进行手术工具分割。CSRT本身获得的EAO分数为0.563。本挑战赛的结果表明,当前非深度学习方法仍具有竞争力。本次挑战赛创建的数据集和基准测试工具已公开发布于https://surgt.grand-challenge.org/。