This paper investigates discrepancies in how neural networks learn from different imaging domains, which are commonly overlooked when adopting computer vision techniques from the domain of natural images to other specialized domains such as medical images. Recent works have found that the generalization error of a trained network typically increases with the intrinsic dimension ($d_{data}$) of its training set. Yet, the steepness of this relationship varies significantly between medical (radiological) and natural imaging domains, with no existing theoretical explanation. We address this gap in knowledge by establishing and empirically validating a generalization scaling law with respect to $d_{data}$, and propose that the substantial scaling discrepancy between the two considered domains may be at least partially attributed to the higher intrinsic "label sharpness" ($K_F$) of medical imaging datasets, a metric which we propose. Next, we demonstrate an additional benefit of measuring the label sharpness of a training set: it is negatively correlated with the trained model's adversarial robustness, which notably leads to models for medical images having a substantially higher vulnerability to adversarial attack. Finally, we extend our $d_{data}$ formalism to the related metric of learned representation intrinsic dimension ($d_{repr}$), derive a generalization scaling law with respect to $d_{repr}$, and show that $d_{data}$ serves as an upper bound for $d_{repr}$. Our theoretical results are supported by thorough experiments with six models and eleven natural and medical imaging datasets over a range of training set sizes. Our findings offer insights into the influence of intrinsic dataset properties on generalization, representation learning, and robustness in deep neural networks.

翻译：本文探究了神经网络在不同成像领域中学习方式的差异，这种差异在将自然图像领域的计算机视觉技术迁移到医学图像等专门领域时经常被忽视。近期研究发现，训练后网络的泛化误差通常随训练集的内在维度($d_{data}$)增加而增大。然而，这种关系的陡峭程度在医学（放射学）和自然成像领域之间存在显著差异，且目前缺乏理论解释。我们通过建立并实证验证关于$d_{data}$的泛化缩放定律来填补这一知识空白，并提出两个领域间显著的缩放差异可能至少部分归因于医学成像数据集较高的内在"标签锐度"($K_F$)，这是我们提出的一个度量指标。接下来，我们展示了测量训练集标签锐度的额外益处：它与训练模型的对抗鲁棒性呈负相关，这导致医学图像模型对抗攻击的脆弱性显著更高。最后，我们将$d_{data}$形式扩展到学习表示内在维度($d_{repr}$)的相关度量，推导出关于$d_{repr}$的泛化缩放定律，并证明$d_{data}$是$d_{repr}$的上界。我们的理论结果通过使用六种模型和十一个自然及医学成像数据集在不同训练集规模下的全面实验得到支持。我们的研究结果揭示了内在数据集属性对深度神经网络泛化、表示学习和鲁棒性的影响。