Deep neural networks are inherently opaque and challenging to interpret. Unlike hand-crafted feature-based models, we struggle to comprehend the concepts learned and how they interact within these models. This understanding is crucial not only for debugging purposes but also for ensuring fairness in ethical decision-making. In our study, we conduct a post-hoc functional interpretability analysis of pretrained speech models using the probing framework [1]. Specifically, we analyze utterance-level representations of speech models trained for various tasks such as speaker recognition and dialect identification. We conduct layer and neuron-wise analyses, probing for speaker, language, and channel properties. Our study aims to answer the following questions: i) what information is captured within the representations? ii) how is it represented and distributed? and iii) can we identify a minimal subset of the network that possesses this information? Our results reveal several novel findings, including: i) channel and gender information are distributed across the network, ii) the information is redundantly available in neurons with respect to a task, iii) complex properties such as dialectal information are encoded only in the task-oriented pretrained network, iv) and is localised in the upper layers, v) we can extract a minimal subset of neurons encoding the pre-defined property, vi) salient neurons are sometimes shared between properties, vii) our analysis highlights the presence of biases (for example gender) in the network. Our cross-architectural comparison indicates that: i) the pretrained models capture speaker-invariant information, and ii) CNN models are competitive with Transformer models in encoding various understudied properties.

翻译：深度神经网络本质上是黑箱模型，难以解释。与基于手工特征的模型不同，我们很难理解这些模型学到了哪些概念以及它们如何相互作用。这种理解不仅对模型调试至关重要，也关系到伦理决策的公平性保障。本研究采用探针框架[1]，对预训练语音模型进行事后功能可解释性分析。具体而言，我们分析了针对说话人识别和方言识别等任务训练的语音模型在语句级表征中的特征，通过逐层和逐神经元分析，探测量化说话人、语言和信道属性。本研究旨在回答以下问题：i) 表征中包含了哪些信息？ii) 这些信息如何表征和分布？iii) 能否识别出网络中承载这些信息的最小神经子集？研究结果揭示了多项新发现，包括：i) 信道和性别信息在网络中分布式存在；ii) 与特定任务相关的神经元中存在信息的冗余编码；iii) 方言属性等复杂特征仅编码在面向任务的预训练网络中，且iv) 集中于网络高层；v) 可提取编码预定属性的最小神经元子集；vi) 关键神经元有时在不同属性间共享；vii) 分析揭示了网络中存在的偏差（如性别偏差）。跨架构比较表明：i) 预训练模型捕获了说话人不变信息；ii) CNN模型在编码各类待研究属性方面与Transformer模型具有竞争力。