Although it has been demonstrated that Natural Language Processing (NLP) algorithms are vulnerable to deliberate attacks, the question of whether such weaknesses can lead to software security threats is under-explored. To bridge this gap, we conducted vulnerability tests on Text-to-SQL systems that are commonly used to create natural language interfaces to databases. We showed that the Text-to-SQL modules within six commercial applications can be manipulated to produce malicious code, potentially leading to data breaches and Denial of Service attacks. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. In addition, experiments using four open-source language models verified that straightforward backdoor attacks on Text-to-SQL systems achieve a 100% success rate without affecting their performance. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms and encourage exploration of methods to mitigate against them.

翻译：尽管已有研究证明自然语言处理（NLP）算法易受蓄意攻击，但此类弱点是否可能导致软件安全威胁的问题尚未得到充分探索。为填补这一空白，我们对常用于构建数据库自然语言接口的文本转SQL系统进行了漏洞测试。研究表明，六个商业应用中的文本转SQL模块可能被操纵生成恶意代码，进而导致数据泄露和拒绝服务攻击。这是首次证明NLP模型可在实际场景中被利用为攻击向量。此外，针对四个开源语言模型的实验证实，针对文本转SQL系统的简单后门攻击可在不影响其性能的前提下实现100%的成功率。本研究旨在引起学界对NLP算法相关潜在软件安全问题的重视，并鼓励探索相应的防御方法。