While language models have made many milestones in text inference and classification tasks, they remain susceptible to adversarial attacks that can lead to unforeseen outcomes. Existing works alleviate this problem by equipping language models with defense patches. However, these defense strategies often rely on impractical assumptions or entail substantial sacrifices in model performance. Consequently, enhancing the resilience of the target model using such defense mechanisms is a formidable challenge. This paper introduces an innovative model for robust text inference and classification, built upon diffusion models (ROIC-DM). Benefiting from its training involving denoising stages, ROIC-DM inherently exhibits greater robustness compared to conventional language models. Moreover, ROIC-DM can attain comparable, and in some cases, superior performance to language models, by effectively incorporating them as advisory components. Extensive experiments conducted with several strong textual adversarial attacks on three datasets demonstrate that (1) ROIC-DM outperforms traditional language models in robustness, even when the latter are fortified with advanced defense mechanisms; (2) ROIC-DM can achieve comparable and even better performance than traditional language models by using them as advisors.

翻译：尽管语言模型在文本推理与分类任务中取得了诸多里程碑式的进展，但其仍容易受到可能导致意外结果的对抗攻击。现有研究通过为语言模型配备防御补丁来缓解这一问题，然而这些防御策略往往基于不切实际的假设，或需要牺牲显著的模型性能。因此，利用此类防御机制增强目标模型的韧性是一项严峻挑战。本文提出了一种基于扩散模型的鲁棒文本推理与分类创新模型（ROIC-DM）。得益于其涉及去噪阶段的训练过程，ROIC-DM相较传统语言模型天然具有更强的鲁棒性。此外，通过将语言模型有效整合为咨询组件，ROIC-DM能够达到甚至在某些情况下超越语言模型的性能。在三个数据集上针对多种强文本对抗攻击的广泛实验表明：（1）即使传统语言模型配备了先进的防御机制，ROIC-DM的鲁棒性仍优于它们；（2）通过将传统语言模型作为顾问，ROIC-DM能够达到甚至取得超越传统语言模型的性能。