A central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called "disentanglement". Most approaches are heuristic and lack a proper theoretical foundation. In linear representation learning, independent component analysis (ICA) has been successful in many applications areas, and it is principled, i.e. based on a well-defined probabilistic model. However, extension of ICA to the nonlinear case has been problematic due to the lack of identifiability, i.e. uniqueness of the representation. Recently, nonlinear extensions that utilize temporal structure or some auxiliary information have been proposed. Such models are in fact identifiable, and consequently, an increasing number of algorithms have been developed. In particular, some self-supervised algorithms can be shown to estimate nonlinear ICA, even though they have initially been proposed from heuristic perspectives. This paper reviews the state-of-the-art of nonlinear ICA theory and algorithms.

翻译：高维数据的有用表示寻找是无监督深度学习中的核心问题，有时称为“分解”。大多数方法是试探性的，缺乏适当的理论基础。在线性表示学习中，独立成分分析（ICA）已经在许多应用领域取得成功，并且是有原则的，即基于一个明确定义的概率模型。然而，将ICA扩展到非线性情况一直是困难的，因为缺乏可识别性，即表征的独特性。最近，已经提出利用时间结构或一些辅助信息的非线性扩展。这些模型实际上是可识别的，因此越来越多的算法已经开发出来。特别地，一些自监督的算法可以被证明是估计非线性ICA，尽管它们最初从试探性的角度提出。本文回顾了非线性ICA理论和算法的最新进展。