This paper investigates discrepancies in how neural networks learn from different imaging domains, which are commonly overlooked when adopting computer vision techniques from the domain of natural images to other specialized domains such as medical images. Recent works have found that the generalization error of a trained network typically increases with the intrinsic dimension ($d_{data}$) of its training set. Yet, the steepness of this relationship varies significantly between medical (radiological) and natural imaging domains, with no existing theoretical explanation. We address this gap in knowledge by establishing and empirically validating a generalization scaling law with respect to $d_{data}$, and propose that the substantial scaling discrepancy between the two considered domains may be at least partially attributed to the higher intrinsic "label sharpness" ($K_F$) of medical imaging datasets, a metric which we propose. Next, we demonstrate an additional benefit of measuring the label sharpness of a training set: it is negatively correlated with the trained model's adversarial robustness, which notably leads to models for medical images having a substantially higher vulnerability to adversarial attack. Finally, we extend our $d_{data}$ formalism to the related metric of learned representation intrinsic dimension ($d_{repr}$), derive a generalization scaling law with respect to $d_{repr}$, and show that $d_{data}$ serves as an upper bound for $d_{repr}$. Our theoretical results are supported by thorough experiments with six models and eleven natural and medical imaging datasets over a range of training set sizes. Our findings offer insights into the influence of intrinsic dataset properties on generalization, representation learning, and robustness in deep neural networks.

翻译：本文研究了神经网络在不同成像领域中学习方式的差异——这一现象在将自然图像领域的计算机视觉技术迁移至医学图像等专业领域时常被忽视。近期研究发现，训练后的网络泛化误差通常随训练集内在维度（$d_{data}$）增加而增大，但医学（放射学）与自然成像领域间该关系的陡峭程度存在显著差异，且缺乏现有理论解释。我们通过建立并实证验证关于$d_{data}$的泛化标度律填补了这一认知空白，并提出两个领域间显著的标度差异可能至少部分归因于医学影像数据集的高内在"标签锐利度"（$K_F$）——这是我们提出的一个度量指标。进一步，我们展示了测量训练集标签锐利度的额外价值：其与训练模型的对抗鲁棒性呈负相关，这尤其导致医学图像模型对对抗攻击的脆弱性显著增高。最后，我们将$d_{data}$的形式化体系扩展至相关度量——学习表征内在维度（$d_{repr}$），推导出关于$d_{repr}$的泛化标度律，并证明$d_{data}$构成$d_{repr}$的上界。通过六种模型及十一个自然与医学成像数据集在不同训练集规模下的全面实验，我们的理论结果得到充分验证。本研究揭示了内在数据集属性对深度神经网络泛化、表征学习及鲁棒性的影响机制。