We apply topological data analysis (TDA) to speech classification problems and to the introspection of a pretrained speech model, HuBERT. To this end, we introduce a number of topological and algebraic features derived from Transformer attention maps and embeddings. We show that a simple linear classifier built on top of such features outperforms a fine-tuned classification head. In particular, we achieve an improvement of about $9\%$ accuracy and $5\%$ ERR on four common datasets; on CREMA-D, the proposed feature set reaches a new state of the art performance with accuracy $80.155$. We also show that topological features are able to reveal functional roles of speech Transformer heads; e.g., we find the heads capable to distinguish between pairs of sample sources (natural/synthetic) or voices without any downstream fine-tuning. Our results demonstrate that TDA is a promising new approach for speech analysis, especially for tasks that require structural prediction. Appendices, an introduction to TDA, and other additional materials are available here - https://topohubert.github.io/speech-topology-webpages/

翻译：我们将拓扑数据分析（TDA）应用于语音分类问题以及预训练语音模型HuBERT的内部剖析。为此，我们引入了从Transformer注意力图和嵌入中提取的一系列拓扑与代数特征。研究表明，基于这些特征构建的简单线性分类器性能优于微调的分类头。具体而言，在四个常用数据集上，我们实现了约9%的准确率提升和5%的错误率降低；在CREMA-D数据集上，所提出的特征集以80.155%的准确率达到了新的最优性能。我们还发现，无需任何下游微调，拓扑特征能够揭示语音Transformer注意力头的功能角色，例如区分样本来源（自然/合成）或说话人声音对。结果表明，TDA是语音分析领域的一种新方法，尤其适用于需要结构预测的任务。附录、TDA入门介绍及其他补充材料可访问 https://topohubert.github.io/speech-topology-webpages/。