Recent years have seen growing interest in learning disentangled representations, in which distinct features, such as size or shape, are represented by distinct neurons. Quantifying the extent to which a given representation is disentangled is not straightforward; multiple metrics have been proposed. In this paper, we identify two failings of existing metrics, which mean they can assign a high score to a model which is still entangled, and we propose two new metrics, which redress these problems. We then consider the task of compositional generalization. Unlike prior works, we treat this as a classification problem, which allows us to use it to measure the disentanglement ability of the encoder, without depending on the decoder. We show that performance on this task is (a) generally quite poor, (b) correlated with most disentanglement metrics, and (c) most strongly correlated with our newly proposed metrics.

翻译：近年来，人们对学习解耦表征的兴趣日益增长，即不同特征（如大小或形状）由不同神经元表示。量化给定表征的解耦程度并非易事；已有多种度量被提出。本文指出了现有度量的两个缺陷，这些缺陷可能导致仍存在纠缠的模型获得高分，并提出了两种新度量以解决这些问题。随后，我们考虑了组合泛化任务。与先前研究不同，我们将此任务视为分类问题，从而可将其用于测量编码器的解耦能力，而无需依赖解码器。我们表明，该任务的性能（a）普遍较差，（b）与大多数解耦度量相关，（c）与我们所提新度量的相关性最强。