Motivated by the recently shown connection between self-attention and (kernel) principal component analysis (PCA), we revisit the fundamentals of PCA. Using the difference-of-convex (DC) framework, we present several novel formulations and provide new theoretical insights. In particular, we show the kernelizability and out-of-sample applicability for a PCA-like family of problems. Moreover, we uncover that simultaneous iteration, which is connected to the classical QR algorithm, is an instance of the difference-of-convex algorithm (DCA), offering an optimization perspective on this longstanding method. Further, we describe new algorithms for PCA and empirically compare them with state-of-the-art methods. Lastly, we introduce a kernelizable dual formulation for a robust variant of PCA that minimizes the $l_1$ deviation of the reconstruction errors.

翻译：受自注意力机制与（核）主成分分析（PCA）之间最新关联的启发，我们重新审视了PCA的基本原理。利用凸差（DC）框架，我们提出了若干新颖的表述并提供了新的理论见解。特别地，我们展示了一类PCA式问题的可核化性及样本外适用性。此外，我们发现与经典QR算法相关联的同步迭代法是凸差算法（DCA）的一个实例，这为这一长期存在的方法提供了优化视角。进一步地，我们描述了新的PCA算法，并通过实证将其与前沿方法进行比较。最后，我们针对PCA的鲁棒变体提出了一种可核化的对偶表述，该变体通过最小化重构误差的$l_1$偏差来提升鲁棒性。