Recent studies reveal that various biases exist in different NLP tasks, and over-reliance on biases results in models' poor generalization ability and low adversarial robustness. To mitigate datasets biases, previous works propose lots of debiasing techniques to tackle specific biases, which perform well on respective adversarial sets but fail to mitigate other biases. In this paper, we propose a new debiasing method Sparse Mixture-of-Adapters (SMoA), which can mitigate multiple dataset biases effectively and efficiently. Experiments on Natural Language Inference and Paraphrase Identification tasks demonstrate that SMoA outperforms full-finetuning, adapter tuning baselines, and prior strong debiasing methods. Further analysis indicates the interpretability of SMoA that sub-adapter can capture specific pattern from the training data and specialize to handle specific bias.

翻译：近期研究表明，不同自然语言处理任务中存在多种偏差，过度依赖偏差会导致模型泛化能力差和对抗鲁棒性低。为缓解数据集偏差，先前工作提出了大量处理特定偏差的去偏技术，这些方法在各自对抗测试集上表现优异，但无法应对其他偏差。本文提出一种新的去偏方法——稀疏混合适配器（Sparse Mixture-of-Adapters, SMoA），能够高效且有效地缓解多种数据集偏差。在自然语言推理和释义识别任务上的实验表明，SMoA优于全参数微调、适配器微调基线以及先前强大的去偏方法。进一步分析揭示了SMoA的可解释性：子适配器能从训练数据中捕获特定模式，并专门处理特定偏差。