A central problem in unsupervised deep learning is how to find useful representations of high-dimensional data, sometimes called "disentanglement". Most approaches are heuristic and lack a proper theoretical foundation. In linear representation learning, independent component analysis (ICA) has been successful in many applications areas, and it is principled, i.e., based on a well-defined probabilistic model. However, extension of ICA to the nonlinear case has been problematic due to the lack of identifiability, i.e., uniqueness of the representation. Recently, nonlinear extensions that utilize temporal structure or some auxiliary information have been proposed. Such models are in fact identifiable, and consequently, an increasing number of algorithms have been developed. In particular, some self-supervised algorithms can be shown to estimate nonlinear ICA, even though they have initially been proposed from heuristic perspectives. This paper reviews the state-of-the-art of nonlinear ICA theory and algorithms.

翻译：无监督深度学习的一个核心问题是如何找到高维数据的有用表示，有时称为“解耦”。大多数方法是启发式的，缺乏适当的理论基础。在线性表示学习中，独立成分分析（ICA）在许多应用领域取得了成功，并且它是基于明确定义的概率模型的原则性方法。然而，由于缺乏可识别性（即表示的唯一性），ICA向非线性情况的扩展一直存在问题。近年来，利用时间结构或某些辅助信息的非线性扩展已被提出。这类模型实际上是可识别的，因此越来越多的算法被开发出来。特别是，一些自监督算法虽然最初是从启发式角度提出的，但可以证明它们能够估计非线性ICA。本文综述了非线性ICA理论和算法的最新进展。