Augmented reality for laparoscopic liver resection is a visualisation mode that allows a surgeon to localise tumours and vessels embedded within the liver by projecting them on top of a laparoscopic image. Preoperative 3D models extracted from CT or MRI data are registered to the intraoperative laparoscopic images during this process. In terms of 3D-2D fusion, most of the algorithms make use of anatomical landmarks to guide registration. These landmarks include the liver's inferior ridge, the falciform ligament, and the occluding contours. They are usually marked by hand in both the laparoscopic image and the 3D model, which is time-consuming and may contain errors if done by a non-experienced user. Therefore, there is a need to automate this process so that augmented reality can be used effectively in the operating room. We present the Preoperative-to-Intraoperative Laparoscopic Fusion Challenge (P2ILF), held during the Medical Imaging and Computer Assisted Interventions (MICCAI 2022) conference, which investigates the possibilities of detecting these landmarks automatically and using them in registration. The challenge was divided into two tasks: 1) A 2D and 3D landmark detection task and 2) a 3D-2D registration task. The teams were provided with training data consisting of 167 laparoscopic images and 9 preoperative 3D models from 9 patients, with the corresponding 2D and 3D landmark annotations. A total of 6 teams from 4 countries participated, whose proposed methods were evaluated on 16 images and two preoperative 3D models from two patients. All the teams proposed deep learning-based methods for the 2D and 3D landmark segmentation tasks and differentiable rendering-based methods for the registration task. Based on the experimental outcomes, we propose three key hypotheses that determine current limitations and future directions for research in this domain.

翻译：腹腔镜肝切除术中的增强现实是一种可视化模式，允许外科医生将嵌入肝脏内的肿瘤和血管投射到腹腔镜图像上，从而定位这些结构。在此过程中，从CT或MRI数据中提取的术前3D模型需与术中腹腔镜图像进行配准。针对3D-2D融合，多数算法利用解剖标志引导配准，这些标志包括肝脏下缘、镰状韧带及遮挡轮廓。这些标志通常需在腹腔镜图像和3D模型中手动标注，既耗时又可能因非经验操作者引入误差。因此，亟需自动化这一过程，以便在手术室中有效应用增强现实技术。我们提出了在医学影像与计算机辅助介入大会（MICCAI 2022）期间举办的"术前到术中腹腔镜融合挑战赛（P2ILF）"，旨在探究自动检测这些标志并用于配准的可能性。该挑战分为两项任务：1) 2D与3D标志检测任务；2) 3D-2D配准任务。参赛团队获得包含来自9位患者的167幅腹腔镜图像及9个术前3D模型在内的训练数据，并附有对应的2D与3D标志标注。共有来自4个国家的6个团队参赛，其方法基于来自两位患者的16幅图像和两个术前3D模型进行评估。所有团队针对2D与3D标志分割任务提出了基于深度学习的方法，针对配准任务则采用基于可微分渲染的方法。基于实验结果，我们提出了三个关键假设，以确定该领域的当前局限性和未来研究方向。