Automated generation of clinically accurate radiology reports can improve patient care. Previous report generation methods that rely on image captioning models often generate incoherent and incorrect text due to their lack of relevant domain knowledge, while retrieval-based attempts frequently retrieve reports that are irrelevant to the input image. In this work, we propose Contrastive X-Ray REport Match (X-REM), a novel retrieval-based radiology report generation module that uses an image-text matching score to measure the similarity of a chest X-ray image and radiology report for report retrieval. We observe that computing the image-text matching score with a language-image model can effectively capture the fine-grained interaction between image and text that is often lost when using cosine similarity. X-REM outperforms multiple prior radiology report generation modules in terms of both natural language and clinical metrics. Human evaluation of the generated reports suggests that X-REM increased the number of zero-error reports and decreased the average error severity compared to the baseline retrieval approach. Our code is available at: https://github.com/rajpurkarlab/X-REM

翻译：自动生成临床准确的放射学报告可改善患者护理。以往依赖图像描述模型的报告生成方法由于缺乏相关领域知识，往往生成不连贯且错误的文本，而基于检索的方法则常检索到与输入图像不相关的报告。本文提出对比性X光报告匹配（X-REM），一种新颖的基于检索的放射学报告生成模块，该模块利用图像-文本匹配评分来衡量胸部X光图像与放射学报告之间的相似性以实现报告检索。我们观察到，使用语言-图像模型计算图像-文本匹配评分，能有效捕捉图像与文本之间常因使用余弦相似度而丢失的细粒度交互。X-REM在自然语言指标和临床指标上均优于多个先前的放射学报告生成模块。对生成报告的人工评估表明，与基线检索方法相比，X-REM增加了零错误报告数量，并降低了平均错误严重程度。我们的代码可在以下地址获取：https://github.com/rajpurkarlab/X-REM