This paper introduces the Intrinsic Dimension Estimating Autoencoder (IDEA), which identifies the underlying intrinsic dimension of a wide range of datasets whose samples lie on either linear or nonlinear manifolds. Beyond estimating the intrinsic dimension, IDEA is also able to reconstruct the original dataset after projecting it onto the corresponding latent space, which is structured using re-weighted double CancelOut layers. Our key contribution is the introduction of the projected reconstruction loss term, guiding the training of the model by continuously assessing the reconstruction quality under the removal of an additional latent dimension. We first assess the performance of IDEA on a series of theoretical benchmarks to validate its robustness. These experiments allow us to test its reconstruction ability and compare its performance with state-of-the-art intrinsic dimension estimators. The benchmarks show good accuracy and high versatility of our approach. Subsequently, we apply our model to data generated from the numerical solution of a vertically resolved one-dimensional free-surface flow, following a pointwise discretization of the vertical velocity profile in the horizontal direction, vertical direction, and time. IDEA succeeds in estimating the dataset's intrinsic dimension and then reconstructs the original solution by working directly within the projection space identified by the network.

翻译：本文提出本征维数估计自编码器（IDEA），该方法能够识别广泛数据集的内在维数，这些数据集的样本分布于线性或非线性流形上。除估计本征维数外，IDEA还能在将数据集投影至相应潜在空间后重建原始数据，该潜在空间通过重新加权的双重CancelOut层构建。我们的核心贡献在于引入投影重建损失项，该损失项通过持续评估在移除一个额外潜在维度下的重建质量来指导模型训练。我们首先在一系列理论基准测试中评估IDEA的性能以验证其鲁棒性。这些实验使我们能够测试其重建能力，并将其性能与最先进的本征维数估计方法进行比较。基准测试表明我们的方法具有良好的准确性和高度的普适性。随后，我们将模型应用于由垂向解析一维自由表面流数值解生成的数据，该数据遵循垂直速度剖面在水平方向、垂直方向及时间上的逐点离散化。IDEA成功估计了数据集的本征维数，并直接在网络识别的投影空间内重建了原始解。