We introduce a novel Region-based contrastive pretraining for Medical Image Retrieval (RegionMIR) that demonstrates the feasibility of medical image retrieval with similar anatomical regions. RegionMIR addresses two major challenges for medical image retrieval i) standardization of clinically relevant searching criteria (e.g., anatomical, pathology-based), and ii) localization of anatomical area of interests that are semantically meaningful. In this work, we propose an ROI image retrieval image network that retrieves images with similar anatomy by extracting anatomical features (via bounding boxes) and evaluate similarity between pairwise anatomy-categorized features between the query and the database of images using contrastive learning. ROI queries are encoded using a contrastive-pretrained encoder that was fine-tuned for anatomy classification, which generates an anatomical-specific latent space for region-correlated image retrieval. During retrieval, we compare the anatomically encoded query to find similar features within a feature database generated from training samples, and retrieve images with similar regions from training samples. We evaluate our approach on both anatomy classification and image retrieval tasks using the Chest ImaGenome Dataset. Our proposed strategy yields an improvement over state-of-the-art pretraining and co-training strategies, from 92.24 to 94.12 (2.03%) classification accuracy in anatomies. We qualitatively evaluate the image retrieval performance demonstrating generalizability across multiple anatomies with different morphology.

翻译：我们提出了一种新颖的基于区域的对比预训练方法，用于医学图像检索（RegionMIR），证明了利用相似解剖区域进行医学图像检索的可行性。RegionMIR解决了医学图像检索的两大挑战：i) 临床相关搜索标准（如解剖学、病理学）的标准化，以及ii) 具有语义意义的感兴趣解剖区域的定位。在本工作中，我们提出了一种感兴趣区域（ROI）图像检索网络，该网络通过提取解剖特征（使用边界框）检索具有相似解剖结构的图像，并利用对比学习评估查询图像与图像数据库之间逐对解剖分类特征之间的相似性。ROI查询通过一个针对解剖分类进行微调的对比预训练编码器进行编码，该编码器生成了一个解剖特异性潜空间，用于区域相关的图像检索。在检索过程中，我们比较解剖编码后的查询与从训练样本生成的特征数据库中的相似特征，并从训练样本中检索具有相似区域的图像。我们使用Chest ImaGenome数据集在解剖分类和图像检索两个任务上评估了我们的方法。所提出的策略在解剖分类准确率上相比当前最优的预训练和协同训练策略从92.24提升到94.12（提升2.03%）。我们定性地评估了图像检索性能，证明了该方法在多种具有不同形态的解剖结构上的泛化能力。