In unsupervised representation learning, models aim to distill essential features from high-dimensional data into lower-dimensional learned representations, guided by inductive biases. Understanding the characteristics that make a good representation remains a topic of ongoing research. Disentanglement of independent generative processes has long been credited with producing high-quality representations. However, focusing solely on representations that adhere to the stringent requirements of most disentanglement metrics, may result in overlooking many high-quality representations, well suited for various downstream tasks. These metrics often demand that generative factors be encoded in distinct, single dimensions aligned with the canonical basis of the representation space. Motivated by these observations, we propose two novel metrics: Importance-Weighted Orthogonality (IWO) and Importance-Weighted Rank (IWR). These metrics evaluate the mutual orthogonality and rank of generative factor subspaces. Throughout extensive experiments on common downstream tasks, over several benchmark datasets and models, IWO and IWR consistently show stronger correlations with downstream task performance than traditional disentanglement metrics. Our findings suggest that representation quality is closer related to the orthogonality of independent generative processes rather than their disentanglement, offering a new direction for evaluating and improving unsupervised learning models.

翻译：在无监督表示学习中，模型的目标是在归纳偏置的引导下，将高维数据中的本质特征提炼至低维学习表示中。理解构成优质表示的特征仍是当前持续研究的课题。长期以来，独立生成过程的解缠一直被认为是产生高质量表示的关键。然而，若仅关注符合大多数解缠度量严格要求的表示，则可能导致忽视许多非常适合各种下游任务的高质量表示。这些度量通常要求生成因子被编码在与表示空间标准基对齐的独立单维中。基于这些观察，我们提出了两个新度量：重要性加权正交性（IWO）和重要性加权秩（IWR）。这些度量评估生成因子子空间的相互正交性与秩。通过在多个基准数据集和模型上进行广泛的下游任务实验，IWO和IWR始终显示出比传统解缠度量更强的与下游任务性能的相关性。我们的研究结果表明，表示质量更接近于与独立生成过程的正交性相关，而非其解缠程度，这为评估和改进无监督学习模型提供了新的方向。