Recent self-supervised contrastive learning methods greatly benefit from the Siamese structure that aims to minimizing distances between positive pairs. These methods usually apply random data augmentation to input images, expecting the augmented views of the same images to be similar and positively paired. However, random augmentation may overlook image semantic information and degrade the quality of augmented views in contrastive learning. This issue becomes more challenging in medical images since the abnormalities related to diseases can be tiny, and are easy to be corrupted (e.g., being cropped out) in the current scheme of random augmentation. In this work, we first demonstrate that, for widely-used X-ray images, the conventional augmentation prevalent in contrastive pre-training can affect the performance of the downstream diagnosis or classification tasks. Then, we propose a novel augmentation method, i.e., FocusContrast, to learn from radiologists' gaze in diagnosis and generate contrastive views for medical images with guidance from radiologists' visual attention. Specifically, we track the gaze movement of radiologists and model their visual attention when reading to diagnose X-ray images. The learned model can predict visual attention of the radiologists given a new input image, and further guide the attention-aware augmentation that hardly neglects the disease-related abnormalities. As a plug-and-play and framework-agnostic module, FocusContrast consistently improves state-of-the-art contrastive learning methods of SimCLR, MoCo, and BYOL by 4.0~7.0% in classification accuracy on a knee X-ray dataset.

翻译：近期，基于孪生结构的自监督对比学习方法通过最小化正样本对之间的距离而显著受益。这类方法通常对输入图像应用随机数据增强，期望同一图像的增强视图具有相似性并构成正样本对。然而，随机增强可能忽略图像语义信息，降低对比学习中增强视图的质量。在医学图像中这一问题尤为严峻，因为与疾病相关的异常区域可能极为微小，且容易在当前随机增强机制下被破坏（例如被裁剪掉）。本文首先证明，对于广泛使用的X光图像，对比预训练中常规采用的增强方式会影响下游诊断或分类任务的性能。随后，我们提出一种新型增强方法——FocusContrast，通过从放射科医师诊断过程中的注视行为中学习，在放射科医师视觉注意力的引导下为医学图像生成对比视图。具体而言，我们追踪放射科医师在阅读X光图像进行诊断时的注视轨迹，并对其视觉注意力进行建模。该学习模型能够针对新输入的图像预测放射科医师的视觉注意力分布，进而引导难以忽略疾病相关异常区域的注意力感知增强。作为即插即用且与框架无关的模块，FocusContrast在膝关节X光数据集上，将SimCLR、MoCo和BYOL等当前最优对比学习方法的分类准确率持续提升4.0%~7.0%。