Deep neural networks have been applied to audio spectrograms for respiratory sound classification. Existing models often treat the spectrogram as a synthetic image while overlooking its physical characteristics. In this paper, a Multi-View Spectrogram Transformer (MVST) is proposed to embed different views of time-frequency characteristics into the vision transformer. Specifically, the proposed MVST splits the mel-spectrogram into different sized patches, representing the multi-view acoustic elements of a respiratory sound. These patches and positional embeddings are then fed into transformer encoders to extract the attentional information among patches through a self-attention mechanism. Finally, a gated fusion scheme is designed to automatically weigh the multi-view features to highlight the best one in a specific scenario. Experimental results on the ICBHI dataset demonstrate that the proposed MVST significantly outperforms state-of-the-art methods for classifying respiratory sounds.

翻译：深度神经网络已被应用于音频频谱图进行呼吸音分类。现有模型常将频谱图视为合成图像，而忽视其物理特性。本文提出一种多视角频谱图变换器（MVST），将时频特性的不同视角嵌入视觉变换器中。具体而言，所提出的MVST将梅尔频谱图分割为不同大小的块，表征呼吸音的多视角声学元素。这些块与位置嵌入随后被输入变换器编码器，通过自注意力机制提取块间的注意力信息。最后，设计门控融合方案自动加权多视角特征，以在特定场景中突出最佳特征。在ICBHI数据集上的实验结果表明，所提出的MVST在呼吸音分类任务上显著优于当前最先进方法。