The goal of automatic report generation is to generate a clinically accurate and coherent phrase from a single given X-ray image, which could alleviate the workload of traditional radiology reporting. However, in a real-world scenario, radiologists frequently face the challenge of producing extensive reports derived from numerous medical images, thereby medical report generation from multi-image perspective is needed. In this paper, we propose the Complex Organ Mask Guided (termed as COMG) report generation model, which incorporates masks from multiple organs (e.g., bones, lungs, heart, and mediastinum), to provide more detailed information and guide the model's attention to these crucial body regions. Specifically, we leverage prior knowledge of the disease corresponding to each organ in the fusion process to enhance the disease identification phase during the report generation process. Additionally, cosine similarity loss is introduced as target function to ensure the convergence of cross-modal consistency and facilitate model optimization.Experimental results on two public datasets show that COMG achieves a 11.4% and 9.7% improvement in terms of BLEU@4 scores over the SOTA model KiUT on IU-Xray and MIMIC, respectively. The code is publicly available at https://github.com/GaryGuTC/COMG_model.

翻译：自动报告生成的目标是从单张给定的X光图像中生成临床准确且连贯的短语，从而减轻传统放射报告的工作负荷。然而，在实际场景中，放射科医生常面临需要根据大量医学图像生成详尽报告的挑战，因此从多图像视角生成医学报告的需求应运而生。本文提出了复杂器官掩膜引导（简称COMG）的报告生成模型。该模型融合了多个器官（如骨骼、肺、心脏和纵隔）的掩膜，以提供更详细的信息，并引导模型关注这些关键身体区域。具体而言，我们在融合过程中利用每个器官对应疾病的先验知识，以增强报告生成过程中的疾病识别阶段。此外，引入余弦相似度损失作为目标函数，以确保跨模态一致性的收敛并促进模型优化。在两个公开数据集上的实验结果表明，与现有最优模型KiUT相比，COMG在IU-Xray和MIMIC数据集上的BLEU@4评分分别提升了11.4%和9.7%。代码已公开于https://github.com/GaryGuTC/COMG_model。