Representation learning is an approach that allows to discover and extract the factors of variation from the data. Intuitively, a representation is said to be disentangled if it separates the different factors of variation in a way that is understandable to humans. Definitions of disentanglement and metrics to measure it usually assume that the factors of variation are independent of each other. However, this is generally false in the real world, which limits the use of these definitions and metrics to very specific and unrealistic scenarios. In this paper we give a definition of disentanglement based on information theory that is also valid when the factors of variation are not independent. Furthermore, we relate this definition to the Information Bottleneck Method. Finally, we propose a method to measure the degree of disentanglement from the given definition that works when the factors of variation are not independent. We show through different experiments that the method proposed in this paper correctly measures disentanglement with non-independent factors of variation, while other methods fail in this scenario.

翻译：表示学习是一种能够从数据中发现并提取变化因子的方法。直观而言，如果一个表示能够以人类可理解的方式分离不同的变化因子，则称该表示是解耦的。现有的解耦定义及其度量指标通常假设变化因子彼此独立。然而，这在现实世界中往往不成立，导致这些定义和指标仅适用于特定且不现实的场景。本文基于信息论提出了一种在变化因子非独立情况下仍然有效的解耦定义，并将该定义与信息瓶颈方法建立理论关联。进一步地，我们提出了一种基于该定义的度量方法，能够在变化因子非独立时量化解耦程度。通过多组实验证明，在变化因子非独立的情况下，本文提出的方法能够准确度量解耦效果，而现有方法在此场景下均失效。