Representation learning is an approach that allows to discover and extract the factors of variation from the data. Intuitively, a representation is said to be disentangled if it separates the different factors of variation in a way that is understandable to humans. Definitions of disentanglement and metrics to measure it usually assume that the factors of variation are independent of each other. However, this is generally false in the real world, which limits the use of these definitions and metrics to very specific and unrealistic scenarios. In this paper we give a definition of disentanglement based on information theory that is also valid when the factors of variation are not independent. Furthermore, we relate this definition to the Information Bottleneck Method. Finally, we propose a method to measure the degree of disentanglement from the given definition that works when the factors of variation are not independent. We show through different experiments that the method proposed in this paper correctly measures disentanglement with non-independent factors of variation, while other methods fail in this scenario.

翻译：表示学习是一种能够从数据中发现并提取变分因子的方法。直观而言，若一个表示能以人类可理解的方式分离不同的变分因子，则称该表示是解耦的。现有的解耦定义及其度量指标通常假设变分因子彼此独立，然而这一假设在现实世界中往往不成立，这导致现有定义和度量方法仅适用于特定且不现实的场景。本文基于信息论提出了一种在变分因子非独立情况下仍然有效的解耦定义，并将该定义与信息瓶颈方法建立理论关联。最后，我们提出一种基于该定义的解耦程度度量方法，适用于变分因子存在依赖关系的场景。通过多组实验验证，本文提出的方法能够准确度量非独立变分因子下的解耦程度，而现有方法在此类场景中均失效。