The analysis of neural representation has become an integral part of research aiming to better understand the inner workings of neural networks. While there are many different approaches to investigate neural representations, an important line of research has focused on doing so through the lens of intrinsic dimensions (IDs). Although this perspective has provided valuable insights and stimulated substantial follow-up research, important limitations of this approach have remained largely unaddressed. In this paper, we highlight a crucial discrepancy between theory and practice of IDs in neural representations, theoretically and empirically showing that common ID estimators are, in fact, not tracking the true underlying ID of the representation. We contrast this negative result with an investigation of the underlying factors that may drive commonly reported ID-related results on neural representation in the literature. Building on these insights, we offer a new perspective on ID estimation in neural representations.

翻译：神经表征分析已成为旨在更好理解神经网络内部机制的研究中不可或缺的一部分。尽管有许多不同方法研究神经表征，一个重要研究方向聚焦于通过本征维度（IDs）的视角进行探索。虽然这一视角提供了有价值的见解并激发了大量后续研究，但其关键局限性在很大程度上仍未得到解决。本文揭示了神经表征中本征维度理论与实践的显著差异，从理论和实证两方面论证常见本征维度估计器实际上并未追踪表征的真实本征维度。针对此负面结果，我们系统探究了可能驱动文献中常见神经表征本征维度相关结果的潜在因素。基于这些见解，我们为神经表征中的本征维度估计提供了新视角。