Purpose: The purpose of this paper is to present a method for real-time 2D-3D non-rigid registration using a single fluoroscopic image. Such a method can find applications in surgery, interventional radiology and radiotherapy. By estimating a three-dimensional displacement field from a 2D X-ray image, anatomical structures segmented in the preoperative scan can be projected onto the 2D image, thus providing a mixed reality view. Methods: A dataset composed of displacement fields and 2D projections of the anatomy is generated from the preoperative scan. From this dataset, a neural network is trained to recover the unknown 3D displacement field from a single projection image. Results: Our method is validated on lung 4D CT data at different stages of the lung deformation. The training is performed on a 3D CT using random (non domain-specific) diffeomorphic deformations, to which perturbations mimicking the pose uncertainty are added. The model achieves a mean TRE over a series of landmarks ranging from 2.3 to 5.5 mm depending on the amplitude of deformation. Conclusion: In this paper, a CNN-based method for real-time 2D-3D non-rigid registration is presented. This method is able to cope with pose estimation uncertainties, making it applicable to actual clinical scenarios, such as lung surgery, where the C-arm pose is planned before the intervention.

翻译：目的：本文旨在提出一种利用单张荧光图像实现实时二维-三维非刚性配准的方法。该方法可应用于外科手术、介入放射学及放射治疗领域。通过从二维X射线图像中估计三维位移场，可将术前扫描中的解剖结构投影到二维图像上，从而提供混合现实视图。方法：基于术前扫描生成由位移场和二维解剖投影组成的数据集。通过该数据集训练神经网络，以从单张投影图像中恢复未知的三维位移场。结果：我们在不同肺变形阶段的肺部四维CT数据上验证了本方法。训练采用三维CT数据进行，并施加随机（非特定领域）微分同胚形变，同时加入模拟位姿不确定性的扰动。模型在一系列标志点上的平均目标配准误差（TRE）范围为2.3至5.5毫米，具体取决于形变幅度。结论：本文提出了一种基于CNN的实时二维-三维非刚性配准方法。该方法能够应对位姿估计的不确定性，从而适用于实际临床场景（如肺外科手术），其中C型臂的位姿在术前即已规划。