Manifold learning flows are a class of generative modelling techniques that assume a low-dimensional manifold description of the data. The embedding of such manifold into the high-dimensional space of the data is achieved via learnable invertible transformations. Therefore, once the manifold is properly aligned via a reconstruction loss, the probability density is tractable on the manifold and maximum likelihood can be used optimize the network parameters. Naturally, the lower-dimensional representation of the data requires an injective-mapping. Recent approaches were able to enforce that density aligns with the modelled manifold, while efficiently calculating the density volume-change term when embedding to the higher-dimensional space. However, unless the injective-mapping is analytically predefined, the learned manifold is not necessarily an efficient representation of the data. Namely, the latent dimensions of such models frequently learn an entangled intrinsic basis with degenerate information being stored in each dimension. Alternatively, if a locally orthogonal and/or sparse basis is to be learned, here coined canonical intrinsic basis, it can serve in learning a more compact latent space representation. Towards this end, we propose a canonical manifold learning flow method, where a novel optimization objective enforces the transformation matrix to have few prominent and orthogonal basis functions. Canonical manifold flow yields a more efficient use of the latent space, automatically generating fewer prominent and distinct dimensions to represent data, and consequently a better approximation of target distributions than other manifold flow methods in most experiments we conducted, resulting in lower FID scores.

翻译：流形学习流是一类基于数据低维流形描述的生成建模技术。这类方法通过可学习的可逆变换，将流形嵌入到数据的高维空间中。通过重构损失使流形正确对齐后，可以在流形上获得可计算的概率密度，并利用最大似然优化网络参数。自然而言，数据的低维表征需要单射映射。近期方法能够在有效计算高维空间嵌入时密度体积变化项的同时，确保密度与建模流形对齐。然而，除非单射映射被解析预定义，否则学习的流形未必是数据的高效表征——这类模型的潜在维度常会学习到纠缠的内在基，导致每个维度存储退化的信息。若学习局部正交和/或稀疏基（此处称为正则化内在基），则可作为更紧凑潜在空间表征的构建基础。为此，我们提出正则化流形学习流方法，通过新颖的优化目标强制变换矩阵具有少量显著的正交基函数。正则化流形流能更高效利用潜在空间，自动生成更少、更显著且相互区分的数据表征维度。在大多数实验中，相较于其他流形流方法，该方法能更精确逼近目标分布，获得更低的FID分数。