In this paper, we seek to design a report generation model that is able to generate reasonable reports even given different images of various body parts. We start by directly merging multiple datasets and training a single report generation model on this one. We, however, observe that the reports generated in such a simple way only obtain comparable performance compared with that trained separately on each specific dataset. We suspect that this is caused by the dilemma between the diversity of body parts and the limited availability of medical data. To develop robust and generalizable models, it is important to consider a diverse range of body parts and medical conditions. However, collecting a sufficiently large dataset for each specific body part can be difficult due to various factors, such as data availability and privacy concerns. Thus, rather than striving for more data, we propose a single-for-multiple (S4M) framework, which seeks to facilitate the learning of the report generation model with two auxiliary priors: an explicit prior (\ie, feeding radiology-informed knowledge) and an implicit prior (\ie, guided by cross-modal features). Specifically, based on the conventional encoder-decoder report generation framework, we incorporate two extra branches: a Radiology-informed Knowledge Aggregation (RadKA) branch and an Implicit Prior Guidance (IPG) branch. We conduct the experiments on our merged dataset which consists of a public dataset (\ie, IU-Xray) and five private datasets, covering six body parts: chest, abdomen, knee, hip, wrist and shoulder. Our S4M model outperforms all the baselines, regardless of whether they are trained on separate or merged datasets. Code is available at: \url{https://github.com/YtongXie/S4M}.

翻译：本文旨在设计一种报告生成模型，即使面对不同身体部位的多类图像，也能生成合理的报告。我们首先尝试直接合并多个数据集，并在该合并数据集上训练单一报告生成模型。然而，我们观察到这种简单方式生成的报告在性能上仅与在各特定数据集上单独训练的模型相当。我们推测这是由于身体部位多样性与医学数据有限性之间的矛盾所致。要开发鲁棒且可泛化的模型，必须考虑不同身体部位和病理状况的多样性，但受数据可用性、隐私限制等因素影响，为每个特定部位收集足够大的数据集十分困难。因此，我们不再追求更多数据，而是提出一种"单一模型适配多部位"（S4M）框架，通过两种辅助先验知识促进报告生成模型的学习：显式先验（即输入放射学知识）与隐式先验（即跨模态特征引导）。具体而言，在传统编码器-解码器报告生成框架基础上，我们引入两个额外分支：放射学知识聚合分支（RadKA）与隐式先验引导分支（IPG）。我们在合并数据集上开展实验，该数据集包含一个公开数据集（IU-Xray）和五个私有数据集，覆盖胸部、腹部、膝盖、髋部、腕部和肩部六个身体部位。无论基线模型是在单独数据集还是合并数据集上训练，我们的S4M模型均优于所有基线方法。代码已开源：\url{https://github.com/YtongXie/S4M}。