This study assesses deep learning models for audio classification in a clinical setting with the constraint of small datasets reflecting real-world prospective data collection. We analyze CNNs, including DenseNet and ConvNeXt, alongside transformer models like ViT, SWIN, and AST, and compare them against pre-trained audio models such as YAMNet and VGGish. Our method highlights the benefits of pre-training on large datasets before fine-tuning on specific clinical data. We prospectively collected two first-of-their-kind patient audio datasets from stroke patients. We investigated various preprocessing techniques, finding that RGB and grayscale spectrogram transformations affect model performance differently based on the priors they learn from pre-training. Our findings indicate CNNs can match or exceed transformer models in small dataset contexts, with DenseNet-Contrastive and AST models showing notable performance. This study highlights the significance of incremental marginal gains through model selection, pre-training, and preprocessing in sound classification; this offers valuable insights for clinical diagnostics that rely on audio classification.

翻译：本研究评估了在临床环境中使用深度学习的音频分类模型，面临小数据集限制（反映现实世界的前瞻性数据采集）。我们分析了包括DenseNet和ConvNeXt在内的CNN，以及ViT、SWIN和AST等Transformer模型，并将其与YAMNet和VGGish等预训练音频模型进行对比。我们的方法凸显了在特定临床数据微调之前先在大规模数据集上进行预训练的益处。我们前瞻性地收集了来自中风患者的两组同类首创的患者音频数据集。我们探索了多种预处理技术，发现RGB和灰度频谱图转换对模型性能的影响因模型从预训练中学习的先验知识而不同。我们的研究结果表明，在小数据场景下，CNN可以媲美或超越Transformer模型，其中DenseNet-Contrastive和AST模型表现尤为突出。本研究强调了在声音分类中通过模型选择、预训练和预处理实现渐进式边际收益的重要性；这为依赖音频分类的临床诊断提供了宝贵见解。