The study of stability and sensitivity of statistical methods or algorithms with respect to their data is an important problem in machine learning and statistics. The performance of the algorithm under resampling of the data is a fundamental way to measure its stability and is closely related to generalization or privacy of the algorithm. In this paper, we study the resampling sensitivity for the principal component analysis (PCA). Given an $ n \times p $ random matrix $ \mathbf{X} $, let $ \mathbf{X}^{[k]} $ be the matrix obtained from $ \mathbf{X} $ by resampling $ k $ randomly chosen entries of $ \mathbf{X} $. Let $ \mathbf{v} $ and $ \mathbf{v}^{[k]} $ denote the principal components of $ \mathbf{X} $ and $ \mathbf{X}^{[k]} $. In the proportional growth regime $ p/n \to \xi \in (0,1] $, we establish the sharp threshold for the sensitivity/stability transition of PCA. When $ k \gg n^{5/3} $, the principal components $ \mathbf{v} $ and $ \mathbf{v}^{[k]} $ are asymptotically orthogonal. On the other hand, when $ k \ll n^{5/3} $, the principal components $ \mathbf{v} $ and $ \mathbf{v}^{[k]} $ are asymptotically colinear. In words, we show that PCA is sensitive to the input data in the sense that resampling even a negligible portion of the input may completely change the output.

翻译：统计方法或算法对其数据稳定性与敏感性的研究是机器学习与统计学中的重要问题。算法在数据重采样下的表现是衡量其稳定性的基本途径，且与算法的泛化性或隐私性密切相关。本文研究了主成分分析（PCA）的重采样敏感性。给定一个$ n \times p $随机矩阵$ \mathbf{X} $，令$ \mathbf{X}^{[k]} $为从$ \mathbf{X} $中随机重采样$ k $个元素后得到的矩阵。设$ \mathbf{v} $和$ \mathbf{v}^{[k]} $分别为$ \mathbf{X} $和$ \mathbf{X}^{[k]} $的主成分。在比例增长机制$ p/n \to \xi \in (0,1] $下，我们建立了PCA敏感性/稳定性转变的锐利阈值。当$ k \gg n^{5/3} $时，主成分$ \mathbf{v} $与$ \mathbf{v}^{[k]} $渐近正交；而当$ k \ll n^{5/3} $时，两者渐近共线。换言之，我们证明PCA对输入数据高度敏感：即使重采样输入中可忽略的一部分，也可能完全改变输出结果。