This study explores using embedding rank as an unsupervised evaluation metric for general-purpose speech encoders trained via self-supervised learning (SSL). Traditionally, assessing the performance of these encoders is resource-intensive and requires labeled data from the downstream tasks. Inspired by the vision domain, where embedding rank has shown promise for evaluating image encoders without tuning on labeled downstream data, this work examines its applicability in the speech domain, considering the temporal nature of the signals. The findings indicate rank correlates with downstream performance within encoder layers across various downstream tasks and for in- and out-of-domain scenarios. However, rank does not reliably predict the best-performing layer for specific downstream tasks, as lower-ranked layers can outperform higher-ranked ones. Despite this limitation, the results suggest that embedding rank can be a valuable tool for monitoring training progress in SSL speech models, offering a less resource-demanding alternative to traditional evaluation methods.

翻译：本研究探讨了将嵌入秩作为一种无监督评估指标，用于评估通过自监督学习（SSL）训练的通用语音编码器。传统上，评估这些编码器的性能需要大量资源，并且依赖于下游任务的标注数据。受视觉领域的启发——在该领域中，嵌入秩已显示出无需在下游标注数据上微调即可评估图像编码器的潜力——本研究考察了该方法在语音领域的适用性，同时考虑了语音信号的时间特性。研究结果表明，在不同下游任务以及领域内和领域外场景中，编码器各层的嵌入秩与其下游性能存在相关性。然而，秩并不能可靠地预测特定下游任务中性能最佳的层，因为较低秩的层有时可能优于较高秩的层。尽管存在这一局限，研究结果仍表明，嵌入秩可以作为一种有价值的工具，用于监测自监督语音模型的训练进展，为传统评估方法提供一种资源需求更低的替代方案。