Knowledge Graph (KG) plays a crucial role in Medical Report Generation (MRG) because it reveals the relations among diseases and thus can be utilized to guide the generation process. However, constructing a comprehensive KG is labor-intensive and its applications on the MRG process are under-explored. In this study, we establish a complete KG on chest X-ray imaging that includes 137 types of diseases and abnormalities. Based on this KG, we find that the current MRG data sets exhibit a long-tailed problem in disease distribution. To mitigate this problem, we introduce a novel augmentation strategy that enhances the representation of disease types in the tail-end of the distribution. We further design a two-stage MRG approach, where a classifier is first trained to detect whether the input images exhibit any abnormalities. The classified images are then independently fed into two transformer-based generators, namely, ``disease-specific generator" and ``disease-free generator" to generate the corresponding reports. To enhance the clinical evaluation of whether the generated reports correctly describe the diseases appearing in the input image, we propose diverse sensitivity (DS), a new metric that checks whether generated diseases match ground truth and measures the diversity of all generated diseases. Results show that the proposed two-stage generation framework and augmentation strategies improve DS by a considerable margin, indicating a notable reduction in the long-tailed problem associated with under-represented diseases.
翻译:知识图谱在医学报告生成中至关重要,因为它揭示了疾病之间的关联,从而可用于指导生成过程。然而,构建全面的知识图谱需要大量人工投入,其在医学报告生成中的应用尚未得到充分探索。本研究建立了包含137种疾病和异常的胸部X光影像完整知识图谱。基于该图谱,我们发现当前医学报告生成数据集在疾病分布上存在长尾问题。为缓解此问题,我们提出了一种新型增强策略,用于增强分布尾端疾病类型的表示。我们进一步设计了一种两阶段医学报告生成方法:首先训练分类器检测输入图像是否异常,然后将分类后的图像分别输入两个基于Transformer的生成器(即“疾病特异性生成器”和“无疾病生成器”)以生成对应报告。为强化临床评估中生成报告是否准确描述输入图像中的疾病,我们提出了多样性灵敏度这一新指标,用于检验生成的疾病是否与真实标注匹配,并衡量所有生成疾病的多样性。实验结果表明,所提出的两阶段生成框架与增强策略显著提升了多样性灵敏度,有效缓解了低代表性疾病相关的长尾问题。