State-of-the-art computer- and robot-assisted surgery systems heavily depend on intraoperative imaging technologies such as CT and fluoroscopy to generate detailed 3D visualization of the patient's anatomy. While imaging techniques are highly accurate, they are based on ionizing radiation and expose patients and clinicians. This study introduces an alternative, radiation-free approach for reconstructing the 3D spine anatomy using RGB-D data. Drawing inspiration from the 3D "mental map" that surgeons form during surgeries, we introduce SurgPointTransformer, a shape completion approach for surgical applications that can accurately reconstruct the unexposed spine regions from sparse observations of the exposed surface. Our method involves two main steps: segmentation and shape completion. The segmentation step includes spinal column localization and segmentation, followed by vertebra-wise segmentation. The segmented vertebra point clouds are then subjected to SurgPointTransformer, which leverages an attention mechanism to learn patterns between visible surface features and the underlying anatomy. For evaluation, we utilize an ex-vivo dataset of nine specimens. Their CT data is used to establish ground truth data that were used to compare to the outputs of our methods. Our method significantly outperforms the state-of-the-art baselines, achieving an average Chamfer Distance of 5.39, an F-Score of 0.85, an Earth Mover's Distance of 0.011, and a Signal-to-Noise Ratio of 22.90 dB. This study demonstrates the potential of our reconstruction method for 3D vertebral shape completion. It enables 3D reconstruction of the entire lumbar spine and surgical guidance without ionizing radiation or invasive imaging. Our work contributes to computer-aided and robot-assisted surgery, advancing the perception and intelligence of these systems.

翻译：最先进的计算机辅助与机器人辅助手术系统高度依赖术中成像技术，如CT和荧光透视，以生成患者解剖结构的详细三维可视化图像。虽然成像技术精度很高，但其基于电离辐射，会使患者和临床医生暴露于辐射之下。本研究提出了一种替代性的、无辐射的方法，利用RGB-D数据重建三维脊柱解剖结构。受外科医生在手术过程中形成的三维"心理地图"启发，我们提出了SurgPointTransformer，一种面向手术应用的形状补全方法，能够从暴露表面的稀疏观测中准确重建未暴露的脊柱区域。我们的方法包含两个主要步骤：分割与形状补全。分割步骤包括脊柱定位与分割，随后进行椎骨级分割。分割后的椎骨点云随后输入SurgPointTransformer，该模型利用注意力机制学习可见表面特征与内部解剖结构之间的模式。为进行评估，我们使用了九个标本的离体数据集。其CT数据用于建立真实值数据，以与我们方法的输出结果进行比较。我们的方法显著优于现有最先进的基线模型，平均倒角距离为5.39，F分数为0.85，推土机距离为0.011，信噪比为22.90 dB。本研究证明了我们的重建方法在三维椎骨形状补全方面的潜力。该方法能够实现整个腰椎的三维重建和手术引导，而无需电离辐射或侵入性成像。我们的工作为计算机辅助和机器人辅助手术做出了贡献，推动了这些系统的感知与智能水平。