Linear principal component analysis (PCA), nonlinear PCA, and linear independent component analysis (ICA) -- those are three methods with single-layer autoencoder formulations for learning linear transformations from data. Linear PCA learns orthogonal transformations (rotations) that orient axes to maximise variance, but it suffers from a subspace rotational indeterminacy: it fails to find a unique rotation for axes that share the same variance. Both nonlinear PCA and linear ICA reduce the subspace indeterminacy from rotational to permutational by maximising statistical independence under the assumption of unit variance. The relationship between all three can be understood by the singular value decomposition of the linear ICA transformation into a sequence of rotation, scale, rotation. Linear PCA learns the first rotation; nonlinear PCA learns the second. The scale is simply the inverse of the standard deviations. The problem is that, in contrast to linear PCA, conventional nonlinear PCA cannot be used directly on the data to learn the first rotation, the first being special as it reduces dimensionality and orders by variances. In this paper, we have identified the cause, and as a solution we propose $\sigma$-PCA: a unified neural model for linear and nonlinear PCA as single-layer autoencoders. One of its key ingredients: modelling not just the rotation but also the scale -- the variances. This model bridges the disparity between linear and nonlinear PCA. And so, like linear PCA, it can learn a semi-orthogonal transformation that reduces dimensionality and orders by variances, but, unlike linear PCA, it does not suffer from rotational indeterminacy.

翻译：线性主成分分析（PCA）、非线性PCA和线性独立成分分析（ICA）是三种基于单层自编码器框架从数据中学习线性变换的方法。线性PCA通过学习正交变换（旋转）来定向坐标轴以最大化方差，但其存在子空间旋转不确定性问题：当多个轴共享相同方差时，无法唯一确定旋转方向。非线性PCA和线性ICA通过在单位方差假设下最大化统计独立性，将子空间不确定性从旋转不确定性降为排列不确定性。三者之间的关系可通过线性ICA变换的奇异值分解（旋转-缩放-旋转序列）来理解：线性PCA学习第一个旋转，非线性PCA学习第二个旋转，缩放因子即为标准差的倒数。问题在于，与线性PCA不同，传统非线性PCA无法直接用于数据以学习第一个旋转——该旋转具有特殊性，因其能降低维度并按方差排序。本文明确了这一问题的成因，并提出σ-PCA作为解决方案：一种将线性和非线性PCA统一为单层自编码器的神经模型。其关键要素之一是不仅建模旋转关系，还同时建模缩放关系（即方差）。该模型弥合了线性PCA与非线性PCA之间的差异。因此，它既能像线性PCA一样学习半正交变换以降低维度并按方差排序，又不同于线性PCA，不会遭受旋转不确定性问题。