Humans have a remarkable ability to disentangle complex sensory inputs (e.g., image, text) into simple factors of variation (e.g., shape, color) without much supervision. This ability has inspired many works that attempt to solve the following question: how do we invert the data generation process to extract those factors with minimal or no supervision? Several works in the literature on non-linear independent component analysis have established this negative result; without some knowledge of the data generation process or appropriate inductive biases, it is impossible to perform this inversion. In recent years, a lot of progress has been made on disentanglement under structural assumptions, e.g., when we have access to auxiliary information that makes the factors of variation conditionally independent. However, existing work requires a lot of auxiliary information, e.g., in supervised classification, it prescribes that the number of label classes should be at least equal to the total dimension of all factors of variation. In this work, we depart from these assumptions and ask: a) How can we get disentanglement when the auxiliary information does not provide conditional independence over the factors of variation? b) Can we reduce the amount of auxiliary information required for disentanglement? For a class of models where auxiliary information does not ensure conditional independence, we show theoretically and experimentally that disentanglement (to a large extent) is possible even when the auxiliary information dimension is much less than the dimension of the true latent representation.

翻译：人类具备一种非凡能力，能够将复杂的感官输入（如图像、文本）解耦为简单的变化因子（如形状、颜色），而无需过多监督。这一能力启发了大量研究工作，试图解决以下问题：如何以最小或无监督的方式逆转数据生成过程以提取这些因子？非线性独立成分分析领域的多项研究已证实这一负面结论：若缺乏对数据生成过程的认知或适当的归纳偏置，则无法实现这种逆转。近年来，在结构性假设下的解耦研究取得了显著进展，例如当我们能够获取使变化因子条件独立的辅助信息时。然而，现有研究需要大量辅助信息，例如在监督分类中要求标签类别数至少等于所有变化因子的总维度。本研究突破这些假设并提出：a) 当辅助信息无法提供变化因子的条件独立性时，如何实现解耦？b) 能否减少解耦所需的辅助信息量？针对一类辅助信息无法确保条件独立的模型，我们从理论和实验上证明：即使辅助信息维度远小于真实潜在表征的维度，解耦（在很大程度上）仍然是可能实现的。