The amount of medical images stored in hospitals is increasing faster than ever; however, utilizing the accumulated medical images has been limited. This is because existing content-based medical image retrieval (CBMIR) systems usually require example images to construct query vectors; nevertheless, example images cannot always be prepared. Besides, there can be images with rare characteristics that make it difficult to find similar example images, which we call isolated samples. Here, we introduce a novel sketch-based medical image retrieval (SBMIR) system that enables users to find images of interest without example images. The key idea lies in feature decomposition of medical images, whereby the entire feature of a medical image can be decomposed into and reconstructed from normal and abnormal features. By extending this idea, our SBMIR system provides an easy-to-use two-step graphical user interface: users first select a template image to specify a normal feature and then draw a semantic sketch of the disease on the template image to represent an abnormal feature. Subsequently, it integrates the two kinds of input to construct a query vector and retrieves reference images with the closest reference vectors. Using two datasets, ten healthcare professionals with various clinical backgrounds participated in the user test for evaluation. As a result, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for isolated samples. Our SBMIR system achieves flexible medical image retrieval on demand, thereby expanding the utility of medical image databases.

翻译：存储在医院的医学图像数量正以前所未有的速度增长，然而，对这些积累的医学图像的有效利用却一直十分有限。这是因为现有的基于内容的医学图像检索（CBMIR）系统通常需要示例图像来构建查询向量，而示例图像并不总是可以获得的。此外，部分图像可能具有罕见特征，导致难以找到相似的示例图像，我们称之为孤立样本。为此，本文提出了一种新颖的基于草图的医学图像检索（SBMIR）系统，该系统使用户能够在没有示例图像的情况下找到感兴趣的图像。其关键思想在于医学图像的特征分解，即医学图像的完整特征可以分解为正常特征和异常特征，并能够通过两者进行重构。通过扩展这一思想，我们提出的SBMIR系统提供了一种易于使用的两步式图形用户界面：用户首先选择一个模板图像来指定正常特征，然后在模板图像上绘制疾病的语义草图来表示异常特征。随后，系统将这两种输入整合起来构建查询向量，并检索出具有最接近参考向量的参考图像。我们使用两个数据集，邀请了十名具有不同临床背景的医疗保健专业人员参与用户测试以进行系统评估。结果表明，我们提出的SBMIR系统能够帮助用户克服之前的挑战，包括基于细粒度图像特征的检索、无示例图像的检索以及针对孤立样本的检索。该系统实现了按需的灵活医学图像检索，从而扩展了医学图像数据库的实用价值。