Adversarial attacks against NLP Deep Learning models are a significant concern. In particular, adversarial samples exploit the model's sensitivity to small input changes. While these changes appear insignificant on the semantics of the input sample, they result in significant decay in model performance. In this paper, we propose Targeted Paraphrasing via RL (TPRL), an approach to automatically learn a policy to generate challenging samples that most likely improve the model's performance. TPRL leverages FLAN T5, a language model, as a generator and employs a self learned policy using a proximal policy gradient to generate the adversarial examples automatically. TPRL's reward is based on the confusion induced in the classifier, preserving the original text meaning through a Mutual Implication score. We demonstrate and evaluate TPRL's effectiveness in discovering natural adversarial attacks and improving model performance through extensive experiments on four diverse NLP classification tasks via Automatic and Human evaluation. TPRL outperforms strong baselines, exhibits generalizability across classifiers and datasets, and combines the strengths of language modeling and reinforcement learning to generate diverse and influential adversarial examples.

翻译：对抗性攻击对NLP深度学习模型构成重大威胁。具体而言，对抗样本利用模型对微小输入变化的敏感性。尽管这些变化在输入样本语义层面看似无关紧要，却会导致模型性能显著下降。本文提出基于强化学习的目标性释义方法（TPRL），该框架可自动学习生成最可能提升模型性能的挑战性样本策略。TPRL利用语言模型FLAN T5作为生成器，采用基于近端策略优化的自学习策略自动生成对抗样本。其奖励机制基于分类器产生的混淆程度，并通过互蕴含分数保持原始文本语义。通过自动评估与人工评估，我们在四项不同NLP分类任务上进行了广泛实验，验证了TPRL在发现自然对抗攻击与提升模型性能方面的有效性。TPRL不仅超越了强基线方法，展现出跨分类器与数据集的泛化能力，更通过融合语言建模与强化学习的优势，生成了多样且具有影响力的对抗样本。