While language models have made many milestones in text inference and classification tasks, they remain susceptible to adversarial attacks that can lead to unforeseen outcomes. Existing works alleviate this problem by equipping language models with defense patches. However, these defense strategies often rely on impractical assumptions or entail substantial sacrifices in model performance. Consequently, enhancing the resilience of the target model using such defense mechanisms is a formidable challenge. This paper introduces an innovative model for robust text inference and classification, built upon diffusion models (ROIC-DM). Benefiting from its training involving denoising stages, ROIC-DM inherently exhibits greater robustness compared to conventional language models. Moreover, ROIC-DM can attain comparable, and in some cases, superior performance to language models, by effectively incorporating them as advisory components. Extensive experiments conducted with several strong textual adversarial attacks on three datasets demonstrate that (1) ROIC-DM outperforms traditional language models in robustness, even when the latter are fortified with advanced defense mechanisms; (2) ROIC-DM can achieve comparable and even better performance than traditional language models by using them as advisors.

翻译：尽管语言模型在文本推理与分类任务中取得了诸多里程碑式的进展，但它们仍容易受到对抗攻击的影响，导致不可预见的后果。现有研究通过为语言模型配备防御补丁来缓解这一问题。然而，这些防御策略通常依赖于不切实际的假设，或需要以模型性能的大幅牺牲为代价。因此，利用此类防御机制增强目标模型的鲁棒性是一项艰巨挑战。本文提出了一种基于扩散模型的新型鲁棒文本推理与分类模型（ROIC-DM）。得益于其训练过程涉及去噪阶段，ROIC-DM天然地比传统语言模型具有更强的鲁棒性。此外，通过有效将其作为咨询组件，ROIC-DM能够达到与语言模型相当甚至更优的性能。在三个数据集上使用多种强文本对抗攻击进行的大量实验表明：（1）即使传统语言模型配备了先进的防御机制，ROIC-DM在鲁棒性上仍优于它们；（2）通过将传统语言模型作为顾问，ROIC-DM能够实现与之相当甚至更好的性能。