Intelligent medical diagnosis has shown remarkable progress based on the large-scale datasets with precise annotations. However, fewer labeled images are available due to significantly expensive cost for annotating data by experts. To fully exploit the easily available unlabeled data, we propose a novel Spatio-Temporal Structure Consistent (STSC) learning framework. Specifically, a gram matrix is derived to combine the spatial structure consistency and temporal structure consistency together. This gram matrix captures the structural similarity among the representations of different training samples. At the spatial level, our framework explicitly enforces the consistency of structural similarity among different samples under perturbations. At the temporal level, we consider the consistency of the structural similarity in different training iterations by digging out the stable sub-structures in a relation graph. Experiments on two medical image datasets (i.e., ISIC 2018 challenge and ChestX-ray14) show that our method outperforms state-of-the-art SSL methods. Furthermore, extensive qualitative analysis on the Gram matrices and heatmaps by Grad-CAM are presented to validate the effectiveness of our method.

翻译：智能医学诊断基于大规模精确标注数据集已取得显著进展。然而，由于专家标注数据成本高昂，可用标注图像数量有限。为充分利用易获取的无标注数据，我们提出了一种新颖的时空结构一致性（STSC）学习框架。具体而言，通过推导格拉姆矩阵将空间结构一致性与时间结构一致性相结合，该矩阵捕捉不同训练样本表征之间的结构相似性。在空间层面，我们的框架显式约束扰动条件下不同样本的结构相似性一致性；在时间层面，通过挖掘关系图中的稳定子结构，考虑不同训练迭代中结构相似性的一致性。在两个医学图像数据集（ISIC 2018挑战赛和ChestX-ray14）上的实验表明，我们的方法优于现有最先进的半监督学习方法。此外，通过格拉姆矩阵和Grad-CAM热图的广泛定性分析，验证了该方法的有效性。