In computer vision, convolutional neural networks (CNN) such as ConvNeXt, have been able to surpass state-of-the-art transformers, partly thanks to depthwise separable convolutions (DSC). DSC, as an approximation of the regular convolution, has made CNNs more efficient in time and memory complexity without deteriorating their accuracy, and sometimes even improving it. In this paper, we first implement DSC into the Pretrained Audio Neural Networks (PANN) family for audio classification on AudioSet, to show its benefits in terms of accuracy/model size trade-off. Second, we adapt the now famous ConvNeXt model to the same task. It rapidly overfits, so we report on techniques that improve the learning process. Our best ConvNeXt model reached 0.471 mean-average precision on AudioSet, which is better than or equivalent to recent large audio transformers, while using three times less parameters. We also achieved positive results in audio captioning and audio retrieval with this model. Our PyTorch source code and checkpoint models are available at https://github.com/topel/audioset-convnext-inf.

翻译：在计算机视觉领域，诸如ConvNeXt等卷积神经网络已能够超越先进Transformer模型，部分原因归功于深度可分离卷积。深度可分离卷积作为常规卷积的近似方法，使CNN在时间和内存复杂度上更高效，同时不降低精度，甚至有时能提升性能。本文首先将深度可分离卷积应用于面向AudioSet音频分类的预训练音频神经网络系列，以展示其在精度与模型规模权衡方面的优势。其次，我们将当前著名的ConvNeXt模型适配至同一任务。该模型迅速出现过拟合现象，因此我们报告了改进学习过程的技术。我们最优的ConvNeXt模型在AudioSet上达到0.471的平均精度均值，优于或等同于近期大型音频Transformer，而参数量仅为后者的三分之一。该模型在音频描述和音频检索中也取得了积极成果。我们的PyTorch源代码和检查点模型可从https://github.com/topel/audioset-convnext-inf获取。