Principal Component Analysis (PCA) is a fundamental tool for data visualization, denoising, and dimensionality reduction. It is widely popular in Statistics, Machine Learning, Computer Vision, and related fields. However, PCA is well-known to fall prey to outliers and often fails to detect the true underlying low-dimensional structure within the dataset. Following the Median of Means (MoM) philosophy, recent supervised learning methods have shown great success in dealing with outlying observations without much compromise to their large sample theoretical properties. This paper proposes a PCA procedure based on the MoM principle. Called the \textbf{M}edian of \textbf{M}eans \textbf{P}rincipal \textbf{C}omponent \textbf{A}nalysis (MoMPCA), the proposed method is not only computationally appealing but also achieves optimal convergence rates under minimal assumptions. In particular, we explore the non-asymptotic error bounds of the obtained solution via the aid of the Rademacher complexities while granting absolutely no assumption on the outlying observations. The derived concentration results are not dependent on the dimension because the analysis is conducted in a separable Hilbert space, and the results only depend on the fourth moment of the underlying distribution in the corresponding norm. The proposal's efficacy is also thoroughly showcased through simulations and real data applications.
翻译:主成分分析(PCA)是数据可视化、去噪和降维的基础工具,广泛应用于统计学、机器学习、计算机视觉及相关领域。然而,PCA易受异常值影响,常无法检测数据集中真正的潜在低维结构。遵循均值中位数(MoM)原则,近期监督学习方法在处理异常观测值时取得了显著成功,且未过多牺牲其大样本理论性质。本文提出一种基于MoM原则的PCA过程,称为均值中位数主成分分析(MoMPCA)。该方法不仅计算简便,而且在最小假设下实现了最优收敛速率。特别地,我们借助Rademacher复杂度推导了所得解的非渐近误差界,且对异常观测值完全不设任何假设。由于分析在可分离希尔伯特空间中进行,导出的集中结果不依赖于维度,仅依赖于对应范数下底层分布的四阶矩。通过模拟实验和实际数据应用,该方法的有效性也得到了全面展示。