In this work, we explore Parameter-Efficient-Learning (PEL) techniques to repurpose a General-Purpose-Speech (GSM) model for Arabic dialect identification (ADI). Specifically, we investigate different setups to incorporate trainable features into a multi-layer encoder-decoder GSM formulation under frozen pre-trained settings. Our architecture includes residual adapter and model reprogramming (input-prompting). We design a token-level label mapping to condition the GSM for Arabic Dialect Identification (ADI). This is challenging due to the high variation in vocabulary and pronunciation among the numerous regional dialects. We achieve new state-of-the-art accuracy on the ADI-17 dataset by vanilla fine-tuning. We further reduce the training budgets with the PEL method, which performs within 1.86% accuracy to fine-tuning using only 2.5% of (extra) network trainable parameters. Our study demonstrates how to identify Arabic dialects using a small dataset and limited computation with open source code and pre-trained models.

翻译：本研究探索了参数高效学习技术，用于将通用语音模型重新应用于阿拉伯方言识别任务。具体而言，我们在冻结预训练参数的设定下，研究了将可训练特征融入多层编码器-解码器通用语音模型的不同配置方案。所提出的架构包含残差适配器与模型重编程（输入提示）机制。我们设计了令牌级标签映射方法，以约束通用语音模型对阿拉伯方言进行识别。由于不同区域方言在词汇和发音上存在显著差异，该任务具有挑战性。通过标准微调方法，我们在ADI-17数据集上取得了新的最优准确率。采用参数高效学习技术后，仅使用2.5%的网络额外可训练参数即可达到与微调方法相差1.86%的准确率，显著降低了训练成本。本研究证明了如何利用小规模数据集和有限计算资源实现阿拉伯方言识别，并提供了开源代码与预训练模型。