The pandemic resulted in vast repositories of unstructured data, including radiology reports, due to increased medical examinations. Previous research on automated diagnosis of COVID-19 primarily focuses on X-ray images, despite their lower precision compared to computed tomography (CT) scans. In this work, we leverage unstructured data from a hospital and harness the fine-grained details offered by CT scans to perform zero-shot multi-label classification based on contrastive visual language learning. In collaboration with human experts, we investigate the effectiveness of multiple zero-shot models that aid radiologists in detecting pulmonary embolisms and identifying intricate lung details like ground glass opacities and consolidations. Our empirical analysis provides an overview of the possible solutions to target such fine-grained tasks, so far overlooked in the medical multimodal pretraining literature. Our investigation promises future advancements in the medical image analysis community by addressing some challenges associated with unstructured data and fine-grained multi-label classification.

翻译：疫情期间，医学检查量激增，产生了包括放射学报告在内的大量非结构化数据。此前针对COVID-19自动诊断的研究主要聚焦于X光影像，尽管其精度低于计算机断层扫描（CT）。本研究利用医院中的非结构化数据，借助CT扫描提供的精细细节，基于对比视觉语言学习实现零样本多标签分类。通过与人类专家合作，我们评估了多种零样本模型在辅助放射科医生检测肺栓塞、识别磨玻璃影和实变等复杂肺部细节方面的有效性。我们的实证分析概述了解决此类精细任务的可能方案，而这些方案在医学多模态预训练文献中至今仍鲜有涉足。该研究通过解决与非结构化数据和细粒度多标签分类相关的一些挑战，有望推动医学图像分析领域的未来发展。