Deep learning has had tremendous success at learning low-dimensional representations of high-dimensional data. This success would be impossible if there was no hidden low-dimensional structure in data of interest; this existence is posited by the manifold hypothesis, which states that the data lies on an unknown manifold of low intrinsic dimension. In this paper, we argue that this hypothesis does not properly capture the low-dimensional structure typically present in image data. Assuming that data lies on a single manifold implies intrinsic dimension is identical across the entire data space, and does not allow for subregions of this space to have a different number of factors of variation. To address this deficiency, we consider the union of manifolds hypothesis, which states that data lies on a disjoint union of manifolds of varying intrinsic dimensions. We empirically verify this hypothesis on commonly-used image datasets, finding that indeed, observed data lies on a disconnected set and that intrinsic dimension is not constant. We also provide insights into the implications of the union of manifolds hypothesis in deep learning, both supervised and unsupervised, showing that designing models with an inductive bias for this structure improves performance across classification and generative modelling tasks. Our code is available at https://github.com/layer6ai-labs/UoMH.

翻译：深度学习在学习高维数据的低维表示方面取得了巨大成功。如果感兴趣的数据中不存在隐藏的低维结构，这一成功将不可能实现；这种存在性由流形假设所假定，该假设认为数据位于一个未知的低固有维度的流形上。在本文中，我们论证该假设并未准确捕捉图像数据中通常存在的低维结构。假设数据位于单一流形上意味着整个数据空间的固有维度是相同的，并且不允许该空间的子区域拥有不同数量的变异因子。为解决这一缺陷，我们考虑了流形并集假设，该假设认为数据位于一个具有不同固有维度的流形的不相交并集上。我们通过实验在常用的图像数据集上验证了这一假设，发现观测数据确实位于一个不连通集合上，且固有维度并非恒定。我们还探讨了流形并集假设在监督与非监督深度学习中的影响，表明设计具有对该结构归纳偏好的模型可提升分类与生成建模任务的性能。我们的代码可在 https://github.com/layer6ai-labs/UoMH 获取。