The success of algorithms in the analysis of high-dimensional data is often attributed to the manifold hypothesis, which supposes that this data lie on or near a manifold of much lower dimension. It is often useful to determine or estimate the dimension of this manifold before performing dimension reduction, for instance. Existing methods for dimension estimation are calibrated using a flat unit ball. In this paper, we develop CA-PCA, a version of local PCA based instead on a calibration of a quadratic embedding, acknowledging the curvature of the underlying manifold. Numerous careful experiments show that this adaptation improves the estimator in a wide range of settings.
翻译:高维数据分析中算法的成功常归因于流形假设,即这些数据位于或邻近一个维度低得多的流形上。例如,在执行降维操作前,确定或估计该流形的维数往往很有必要。现有的维数估计方法均以平坦单位球为校准基准。本文提出CA-PCA方法,这是一种基于二次嵌入校准的局部主成分分析变体,通过承认底层流形的曲率进行改进。大量精细实验表明,这种适应性改进在多种场景下均能提升估计器的性能。