Linear feature extraction at the presence of nonlinear dependencies among the data is a fundamental challenge in unsupervised learning. We propose using a Probabilistic Gram-Schmidt (PGS) type orthogonalization process in order to detect and map out redundant dimensions. Specifically, by applying the PGS process over any family of functions which presumably captures the nonlinear dependencies in the data, we construct a series of covariance matrices that can either be used to remove those dependencies from the principal components, or to identify new large-variance directions. In the former case, we prove that under certain assumptions the resulting algorithms detect and remove nonlinear dependencies whenever those dependencies lie in the linear span of the chosen function family. In the latter, we provide information-theoretic guarantees in terms of entropy reduction. Both proposed methods extract linear features from the data while removing nonlinear redundancies. We provide simulation results on synthetic and real-world datasets which show improved performance over PCA and state-of-the-art linear feature extraction algorithms, both in terms of variance maximization of the extracted features, and in terms of improved performance of classification algorithms.

翻译：在数据存在非线性依赖关系的情况下进行线性特征提取是无监督学习中的一个基本挑战。我们提出使用一种概率化Gram-Schmidt（PGS）类型的正交化过程来检测并映射出冗余维度。具体而言，通过对任意一组假设能捕捉数据中非线性依赖关系的函数族应用PGS过程，我们构造了一系列协方差矩阵，这些矩阵既可以用于从主成分中移除这些依赖关系，也可以用于识别新的高方差方向。在前一种情形下，我们证明在特定假设下，当这些非线性依赖关系位于所选函数族的线性张成空间内时，所提出的算法能够检测并移除它们。在后一种情形下，我们提供了基于熵减的信息论保证。这两种方法在移除非线性冗余的同时从数据中提取线性特征。我们在合成数据集和真实世界数据集上给出了仿真结果，这些结果表明：在提取特征的方差最大化以及分类算法性能提升方面，所提方法均优于主成分分析（PCA）及现有最优的线性特征提取算法。