Medical report generation is one of the most challenging tasks in medical image analysis. Although existing approaches have achieved promising results, they either require a predefined template database in order to retrieve sentences or ignore the hierarchical nature of medical report generation. To address these issues, we propose MedWriter that incorporates a novel hierarchical retrieval mechanism to automatically extract both report and sentence-level templates for clinically accurate report generation. MedWriter first employs the Visual-Language Retrieval~(VLR) module to retrieve the most relevant reports for the given images. To guarantee the logical coherence between sentences, the Language-Language Retrieval~(LLR) module is introduced to retrieve relevant sentences based on the previous generated description. At last, a language decoder fuses image features and features from retrieved reports and sentences to generate meaningful medical reports. We verified the effectiveness of our model by automatic evaluation and human evaluation on two datasets, i.e., Open-I and MIMIC-CXR.

翻译：医学报告生成是医学图像分析中最具挑战性的任务之一。虽然现有方法已经取得了可喜的成果,但它们要么需要预先定义的模板数据库,以便检索判决,要么忽视医学报告生成的等级性质。为了解决这些问题,我们建议梅德怀特采用新的等级检索机制,以自动提取报告和句级模板,用于生成临床准确的报告。麦德怀特首先使用视觉语言检索检索~(VLR)模块,为给定图像检索最相关的报告。为了保证判决之间的逻辑一致性,根据先前生成的描述引入了语言语言检索访问~(LLLLR)模块,以检索相关句子。最后,语言解密器连接了检索报告和句子的图像特征和特征,以产生有意义的医学报告。我们通过对两个数据集,即开放一和MIMIC-CXR进行自动评估和人文评估,验证了我们模型的有效性。