This paper investigates discrepancies in how neural networks learn from different imaging domains, which are commonly overlooked when adopting computer vision techniques from the domain of natural images to other specialized domains such as medical images. Recent works have found that the generalization error of a trained network typically increases with the intrinsic dimension ($d_{data}$) of its training set. Yet, the steepness of this relationship varies significantly between medical (radiological) and natural imaging domains, with no existing theoretical explanation. We address this gap in knowledge by establishing and empirically validating a generalization scaling law with respect to $d_{data}$, and propose that the substantial scaling discrepancy between the two considered domains may be at least partially attributed to the higher intrinsic ``label sharpness'' ($K_\mathcal{F}$) of medical imaging datasets, a metric which we propose. Next, we demonstrate an additional benefit of measuring the label sharpness of a training set: it is negatively correlated with the trained model's adversarial robustness, which notably leads to models for medical images having a substantially higher vulnerability to adversarial attack. Finally, we extend our $d_{data}$ formalism to the related metric of learned representation intrinsic dimension ($d_{repr}$), derive a generalization scaling law with respect to $d_{repr}$, and show that $d_{data}$ serves as an upper bound for $d_{repr}$. Our theoretical results are supported by thorough experiments with six models and eleven natural and medical imaging datasets over a range of training set sizes. Our findings offer insights into the influence of intrinsic dataset properties on generalization, representation learning, and robustness in deep neural networks. Code link: https://github.com/mazurowski-lab/intrinsic-properties

翻译：本文研究了神经网络从不同成像域学习时存在的差异，这些差异在将自然图像领域的计算机视觉技术迁移至医学图像等其他专业领域时常被忽视。近期研究发现，训练网络的泛化误差通常随训练集内在维度（$d_{data}$）的增加而增大。然而，该关系的陡峭程度在医学（放射学）与自然成像域之间存在显著差异，且尚无现有理论解释。为弥补这一认知空白，我们建立并实证验证了关于$d_{data}$的泛化缩放定律，并提出两个领域间的显著缩放差异可能至少部分归因于医学成像数据集具有更高的内在“标签锐度”（$K_\mathcal{F}$）——一项我们提出的新度量指标。继而，我们展示了测量训练集标签锐度的额外效益：该指标与训练模型的对抗鲁棒性呈负相关，这尤其导致医学图像模型对对抗攻击的脆弱性显著更高。最后，我们将$d_{data}$形式化框架扩展至学习表示内在维度（$d_{repr}$）的关联度量，推导出关于$d_{repr}$的泛化缩放定律，并证明$d_{data}$是$d_{repr}$的上界。我们的理论结果通过涵盖六种模型及十一组自然与医学成像数据集（覆盖多种训练集规模）的全面实验得到支持。研究成果揭示了数据集内在属性对深度神经网络泛化能力、表示学习及鲁棒性的影响机制。代码链接：https://github.com/mazurowski-lab/intrinsic-properties