The application of eye-tracking techniques in medical image analysis has become increasingly popular in recent years. It collects the visual search patterns of the domain experts, containing much important information about health and disease. Therefore, how to efficiently integrate radiologists' gaze patterns into the diagnostic analysis turns into a critical question. Existing works usually transform gaze information into visual attention maps (VAMs) to supervise the learning process. However, this time-consuming procedure makes it difficult to develop end-to-end algorithms. In this work, we propose a novel gaze-guided graph neural network (GNN), GazeGNN, to perform disease classification from medical scans. In GazeGNN, we create a unified representation graph that models both the image and gaze pattern information. Hence, the eye-gaze information is directly utilized without being converted into VAMs. With this benefit, we develop a real-time, real-world, end-to-end disease classification algorithm for the first time and avoid the noise and time consumption introduced during the VAM preparation. To our best knowledge, GazeGNN is the first work that adopts GNN to integrate image and eye-gaze data. Our experiments on the public chest X-ray dataset show that our proposed method exhibits the best classification performance compared to existing methods.

翻译：摘要：近年来，眼动追踪技术在医学图像分析中的应用日益普及。该技术收集领域专家的视觉搜索模式，其中包含大量与健康和疾病相关的重要信息。因此，如何有效整合放射科医生的注视模式到诊断分析中成为一个关键问题。现有工作通常将注视信息转换为视觉注意力图（VAM）来监督学习过程。然而，这种耗时的方法使得开发端到端算法变得困难。在本工作中，我们提出了一种新颖的基于注视引导的图神经网络（GNN）——GazeGNN，用于从医学扫描中执行疾病分类。在GazeGNN中，我们构建了一个统一表示图，同时建模图像和注视模式信息。由此，眼动注视信息无需转换为VAM即可直接使用。利用这一优势，我们首次开发了一种实时的、面向真实世界的端到端疾病分类算法，并避免了VAM准备过程中引入的噪声和时间消耗。据我们所知，GazeGNN是首个采用GNN整合图像与眼动注视数据的工作。我们在公共胸部X光数据集上的实验表明，与现有方法相比，我们所提出的方法展现出最佳的分类性能。