Although it has been demonstrated that Natural Language Processing (NLP) algorithms are vulnerable to deliberate attacks, the question of whether such weaknesses can lead to software security threats is under-explored. To bridge this gap, we conducted vulnerability tests on Text-to-SQL systems that are commonly used to create natural language interfaces to databases. We showed that the Text-to-SQL modules within six commercial applications can be manipulated to produce malicious code, potentially leading to data breaches and Denial of Service attacks. This is the first demonstration that NLP models can be exploited as attack vectors in the wild. In addition, experiments using four open-source language models verified that straightforward backdoor attacks on Text-to-SQL systems achieve a 100% success rate without affecting their performance. The aim of this work is to draw the community's attention to potential software security issues associated with NLP algorithms and encourage exploration of methods to mitigate against them.

翻译：尽管已有研究表明自然语言处理算法易受蓄意攻击，但此类缺陷是否会导致软件安全威胁的问题尚未得到充分探索。为填补这一空白，我们对常用于构建数据库自然语言接口的文本到SQL系统进行了脆弱性测试。研究表明，六款商业应用中的文本到SQL模块可被操纵生成恶意代码，可能导致数据泄露和拒绝服务攻击。这是首次证明自然语言处理模型可在真实环境中被用作攻击向量。此外，使用四款开源语言模型的实验验证，针对文本到SQL系统的直接后门攻击实现了100%的成功率且不影响其性能。本工作旨在引起学界对自然语言处理算法相关潜在软件安全问题的关注，并鼓励探索相应的防御方法。