Medical report generation task, which targets to produce long and coherent descriptions of medical images, has attracted growing research interests recently. Different from the general image captioning tasks, medical report generation is more challenging for data-driven neural models. This is mainly due to 1) the serious data bias and 2) the limited medical data. To alleviate the data bias and make best use of available data, we propose a Competence-based Multimodal Curriculum Learning framework (CMCL). Specifically, CMCL simulates the learning process of radiologists and optimizes the model in a step by step manner. Firstly, CMCL estimates the difficulty of each training instance and evaluates the competence of current model; Secondly, CMCL selects the most suitable batch of training instances considering current model competence. By iterating above two steps, CMCL can gradually improve the model's performance. The experiments on the public IU-Xray and MIMIC-CXR datasets show that CMCL can be incorporated into existing models to improve their performance.

翻译：医学报告生成任务旨在为医学图像生成长且连贯的描述，近年来吸引了越来越多的研究兴趣。与通用图像描述任务不同，医学报告生成对数据驱动的神经模型更具挑战性，这主要源于1）严重的数据偏差和2）有限的医学数据。为缓解数据偏差并充分利用现有数据，我们提出了一种基于能力的多模态课程学习框架（CMCL）。具体而言，CMCL模拟放射科医生的学习过程，以逐步方式优化模型。首先，CMCL评估每个训练实例的难度并衡量当前模型的能力；其次，CMCL根据当前模型能力选择最合适的批训练实例。通过迭代上述两个步骤，CMCL能逐步提升模型性能。在公开的IU-Xray和MIMIC-CXR数据集上的实验表明，CMCL可集成到现有模型中以提高其性能。