State-of-the-art computer- and robot-assisted surgery systems heavily depend on intraoperative imaging technologies such as CT and fluoroscopy to generate detailed 3D visualization of the patient's anatomy. While imaging techniques are highly accurate, they are based on ionizing radiation and expose patients and clinicians. This study introduces an alternative, radiation-free approach for reconstructing the 3D spine anatomy using RGB-D data. Drawing inspiration from the 3D "mental map" that surgeons form during surgeries, we introduce SurgPointTransformer, a shape completion approach for surgical applications that can accurately reconstruct the unexposed spine regions from sparse observations of the exposed surface. Our method involves two main steps: segmentation and shape completion. The segmentation step includes spinal column localization and segmentation, followed by vertebra-wise segmentation. The segmented vertebra point clouds are then subjected to SurgPointTransformer, which leverages an attention mechanism to learn patterns between visible surface features and the underlying anatomy. For evaluation, we utilize an ex-vivo dataset of nine specimens. Their CT data is used to establish ground truth data that were used to compare to the outputs of our methods. Our method significantly outperforms the state-of-the-art baselines, achieving an average Chamfer Distance of 5.39, an F-Score of 0.85, an Earth Mover's Distance of 0.011, and a Signal-to-Noise Ratio of 22.90 dB. This study demonstrates the potential of our reconstruction method for 3D vertebral shape completion. It enables 3D reconstruction of the entire lumbar spine and surgical guidance without ionizing radiation or invasive imaging. Our work contributes to computer-aided and robot-assisted surgery, advancing the perception and intelligence of these systems.

翻译：最先进的计算机辅助和机器人辅助手术系统严重依赖术中成像技术（如CT和荧光透视）来生成患者解剖结构的详细三维可视化。虽然成像技术精度很高，但其基于电离辐射，会使患者和临床医生暴露于辐射之下。本研究提出了一种利用RGB-D数据重建三维脊柱解剖结构的替代性无辐射方法。受外科医生在手术过程中形成的三维"心理图谱"启发，我们提出了SurgPointTransformer——一种用于手术应用的形状补全方法，能够从暴露表面的稀疏观测中准确重建未暴露的脊柱区域。我们的方法包含两个主要步骤：分割与形状补全。分割步骤包括脊柱定位与分割，随后进行椎骨级分割。分割后的椎骨点云数据将输入SurgPointTransformer，该模型通过注意力机制学习可见表面特征与底层解剖结构之间的关联模式。为进行评估，我们使用了九个标本的离体数据集。其CT数据被用于建立真实值数据，以与我们方法的输出结果进行比较。我们的方法显著优于现有基线模型，平均倒角距离达到5.39，F分数为0.85，推土机距离为0.011，信噪比为22.90 dB。本研究证明了我们的重建方法在三维椎骨形状补全方面的潜力。该方法能够在不使用电离辐射或侵入性成像的情况下，实现整个腰椎的三维重建和手术引导。我们的工作为计算机辅助和机器人辅助手术领域做出贡献，推动了这些系统的感知能力与智能化发展。