Despite their success in speech processing, neural networks often operate as black boxes, prompting the question: what informs their decisions, and how can we interpret them? This work examines this issue in the context of lexical stress. A dataset of English disyllabic words was automatically constructed from read and spontaneous speech. Several Convolutional Neural Network (CNN) architectures were trained to predict stress position from a spectrographic representation of disyllabic words lacking minimal stress pairs (e.g., initial stress WAllet, final stress exTEND), achieving up to 92% accuracy on held-out test data. Layerwise Relevance Propagation (LRP), a technique for neural network interpretability analysis, revealed that predictions for held-out minimal pairs (PROtest vs. proTEST ) were most strongly influenced by information in stressed versus unstressed syllables, particularly the spectral properties of stressed vowels. However, the classifiers also attended to information throughout the word. A feature-specific relevance analysis is proposed, and its results suggest that our best-performing classifier is strongly influenced by the stressed vowel's first and second formants, with some evidence that its pitch and third formant also contribute. These results reveal deep learning's ability to acquire distributed cues to stress from naturally occurring data, extending traditional phonetic work based around highly controlled stimuli.

翻译：尽管神经网络在语音处理领域取得了显著成功，但其运作机制往往如同黑箱，这引发了一个关键问题：是什么因素影响了它们的决策？我们又该如何解释这些决策？本研究在词汇重音的语境下探讨了这一问题。我们从朗读语音和自发语音中自动构建了一个英语双音节词数据集。研究训练了多种卷积神经网络（CNN）架构，旨在从缺乏最小重音对（例如，重音在首音节的 WAllet 与重音在尾音节的 exTEND）的双音节词语谱图表示中预测重音位置，并在预留的测试数据上达到了高达92%的准确率。通过使用神经网络可解释性分析技术——层相关性传播（LRP），我们发现，对于预留的最小重音对（PROtest 与 proTEST）的预测，最强的影响因素来自重读音节与非重读音节之间的信息差异，特别是重读元音的频谱特性。然而，分类器也关注了整个单词范围内的信息。我们提出了一种针对特定特征的相关性分析方法，其结果表明，我们性能最佳的分类器主要受到重读元音的第一和第二共振峰的影响，同时也有证据表明其音高和第三共振峰也起到一定作用。这些结果揭示了深度学习能够从自然产生的数据中学习到分布式的重音线索，从而拓展了基于高度受控刺激的传统语音学研究。