Medical vision language pre-training (VLP) has emerged as a frontier of research, enabling zero-shot pathological recognition by comparing the query image with the textual descriptions for each disease. Due to the complex semantics of biomedical texts, current methods struggle to align medical images with key pathological findings in unstructured reports. This leads to the misalignment with the target disease's textual representation. In this paper, we introduce a novel VLP framework designed to dissect disease descriptions into their fundamental aspects, leveraging prior knowledge about the visual manifestations of pathologies. This is achieved by consulting a large language model and medical experts. Integrating a Transformer module, our approach aligns an input image with the diverse elements of a disease, generating aspect-centric image representations. By consolidating the matches from each aspect, we improve the compatibility between an image and its associated disease. Additionally, capitalizing on the aspect-oriented representations, we present a dual-head Transformer tailored to process known and unknown diseases, optimizing the comprehensive detection efficacy. Conducting experiments on seven downstream datasets, ours improves the accuracy of recent methods by up to 8.56% and 17.0% for seen and unseen categories, respectively. Our code is released at https://github.com/HieuPhan33/MAVL.

翻译：医学视觉-语言预训练（VLP）已成为研究前沿，通过将查询图像与每种疾病的文本描述进行对比，能够实现零样本病理识别。由于生物医学文本的语义复杂性，当前方法难以将医学图像与非结构化报告中的关键病理发现对齐，导致与目标疾病文本表征的错配。本文提出一种新型VLP框架，旨在将疾病描述分解为基本方面，利用关于病理视觉表现形式的先验知识。该过程通过咨询大语言模型和医学专家实现。通过集成Transformer模块，我们的方法将输入图像与疾病的不同要素对齐，生成基于角度的图像表征。通过整合每个角度的匹配结果，我们提升了图像与其对应疾病之间的兼容性。此外，基于角度导向的表征，我们提出一种双头Transformer，专门处理已知和未知疾病，优化综合检测效能。在七个下游数据集上的实验表明，对于已知和未知类别，我们的方法分别将最近方法的准确率提升最高达8.56%和17.0%。我们的代码已发布在https://github.com/HieuPhan33/MAVL。