Vision-language models, while effective in general domains and showing strong performance in diverse multi-modal applications like visual question-answering (VQA), struggle to maintain the same level of effectiveness in more specialized domains, e.g., medical. We propose a medical vision-language model that integrates large vision and language models adapted for the medical domain. This model goes through three stages of parameter-efficient training using three separate biomedical and radiology multi-modal visual and text datasets. The proposed model achieves state-of-the-art performance on the SLAKE 1.0 medical VQA (MedVQA) dataset with an overall accuracy of 87.5% and demonstrates strong performance on another MedVQA dataset, VQA-RAD, achieving an overall accuracy of 73.2%.

翻译：视觉-语言模型在通用领域表现出色，并在视觉问答等多模态应用中展现强大性能，但在医学等专业领域难以维持同等效果。我们提出了一种医学视觉-语言模型，该模型集成了面向医学领域适配的大规模视觉与语言模型。该模型通过使用三个独立的生物医学与放射学多模态视觉与文本数据集，经过三阶段参数高效训练。所提模型在SLAKE 1.0医学视觉问答数据集上达到了最先进的性能，总体准确率为87.5%，并在另一个医学视觉问答数据集VQA-RAD上展现了强劲性能，总体准确率达到73.2%。