Unsupervised learning has recently significantly gained in popularity, especially with deep learning-based approaches. Despite numerous successes and approaching supervised-level performance on a variety of academic benchmarks, it is still hard to train and evaluate SSL models in practice due to the unsupervised nature of the problem. Even with networks trained in a supervised fashion, it is often unclear whether they will perform well when transferred to another domain. Past works are generally limited to assessing the amount of information contained in embeddings, which is most relevant for self-supervised learning of deep neural networks. This works chooses to follow a different approach: can we quantify how easy it is to linearly separate the data in a stable way? We survey the literature and uncover three methods that could be potentially used for evaluating quality of representations. We also introduce one novel method based on recent advances in understanding the high-dimensional geometric structure self-supervised learning. We conduct extensive experiments and study the properties of these metrics and ones introduced in the previous work. Our results suggest that while there is no free lunch, there are metrics that can robustly estimate embedding quality in an unsupervised way.

翻译：无监督学习近年来显著流行，尤其是基于深度学习的方法。尽管在多个学术基准上取得了诸多成功并接近监督学习性能，但由于问题的无监督性质，在实践中训练和评估自监督学习（SSL）模型仍然困难。即使使用监督方式训练的网络，在迁移至其他领域时，其性能是否良好也往往不明确。以往的工作通常局限于评估嵌入中包含的信息量，这与深度神经网络的自监督学习最为相关。本研究选择了一条不同的路线：我们能否量化以稳定方式线性分离数据的难易程度？我们调研了文献，发现三种可能用于评估表示质量的方法。同时，我们基于理解自监督学习高维几何结构的最新进展，提出了一种新方法。我们进行了大量实验，研究了这些度量指标以及先前工作中引入的度量指标的性质。结果表明，尽管没有免费午餐，但仍存在一些能够在无监督方式下稳健估计嵌入质量的度量指标。