Radiotherapists require accurate registration of MR/CT images to effectively use information from both modalities. In a typical registration pipeline, rigid or affine transformations are applied to roughly align the fixed and moving images before proceeding with the deformation step. While recent learning-based methods have shown promising results in the rigid/affine step, these methods often require images with similar field-of-view (FOV) for successful alignment. As a result, aligning images with different FOVs remains a challenging task. Self-supervised landmark detection methods like self-supervised Anatomical eMbedding (SAM) have emerged as a useful tool for mapping and cropping images to similar FOVs. However, these methods are currently limited to intra-modality use only. To address this limitation and enable cross-modality matching, we propose a new approach called Cross-SAM. Our approach utilizes a novel iterative process that alternates between embedding learning and CT-MRI registration. We start by applying aggressive contrast augmentation on both CT and MRI images to train a SAM model. We then use this SAM to identify corresponding regions on paired images using robust grid-points matching, followed by a point-set based affine/rigid registration, and a deformable fine-tuning step to produce registered paired images. We use these registered pairs to enhance the matching ability of SAM, which is then processed iteratively. We use the final model for cross-modality matching tasks. We evaluated our approach on two CT-MRI affine registration datasets and found that Cross-SAM achieved robust affine registration on both datasets, significantly outperforming other methods and achieving state-of-the-art performance.

翻译：放射治疗师需精确配准MR/CT图像，以有效利用两种模态的信息。在典型配准流程中，先通过刚性或仿射变换对固定图像与移动图像进行粗略对齐，再执行形变步骤。尽管近期基于学习的方法在刚性/仿射步骤中展现出良好效果，但这些方法通常要求图像具有相似的视野范围（FOV）才能实现成功对齐。因此，对齐不同FOV的图像仍是具有挑战性的任务。自监督解剖嵌入（SAM）等自监督地标检测方法已成为将图像映射并裁剪至相似FOV的有效工具。然而，这类方法目前仅适用于单模态场景。为解决该局限并实现跨模态匹配，我们提出名为Cross-SAM的新方法。该方法采用创新迭代流程，在嵌入学习与CT-MRI配准之间交替进行：首先对CT和MRI图像应用强对比度增强技术训练SAM模型；继而利用该SAM通过鲁棒网格点匹配定位配对图像中的对应区域，依次进行基于点集的仿射/刚性配准与可变形微调步骤，生成配准后的配对图像；随后用这些配准对增强SAM的匹配能力，并循环迭代优化。最终模型用于跨模态匹配任务。我们在两个CT-MRI仿射配准数据集上评估该方法，结果显示Cross-SAM在两组数据上均实现鲁棒仿射配准，显著超越其他方法，达到最优性能。