Language generation models have been an increasingly powerful enabler for many applications. Many such models offer free or affordable API access, which makes them potentially vulnerable to model extraction attacks through distillation. To protect intellectual property (IP) and ensure fair use of these models, various techniques such as lexical watermarking and synonym replacement have been proposed. However, these methods can be nullified by obvious countermeasures such as "synonym randomization". To address this issue, we propose GINSEW, a novel method to protect text generation models from being stolen through distillation. The key idea of our method is to inject secret signals into the probability vector of the decoding steps for each target token. We can then detect the secret message by probing a suspect model to tell if it is distilled from the protected one. Experimental results show that GINSEW can effectively identify instances of IP infringement with minimal impact on the generation quality of protected APIs. Our method demonstrates an absolute improvement of 19 to 29 points on mean average precision (mAP) in detecting suspects compared to previous methods against watermark removal attacks.

翻译：语言生成模型已成为众多应用日益强大的推动力。许多此类模型提供免费或负担得起的API访问，这使得它们可能因蒸馏而遭受模型提取攻击。为了保护知识产权（IP）并确保这些模型的公平使用，研究者提出了多种技术，如词汇水印和同义词替换。然而，这些方法可能被“同义词随机化”等明显对策所抵消。为解决此问题，我们提出GINSEW，一种新颖的保护文本生成模型免受蒸馏盗窃的方法。该方法的核心思想是将秘密信号注入每个目标词解码步骤的概率向量中。随后，我们可通过探测可疑模型来检测秘密消息，以判断其是否是从受保护模型蒸馏而来。实验结果表明，GINSEW能有效识别知识产权侵权实例，同时对受保护API的生成质量影响极小。与以往方法相比，我们的方法在检测可疑模型时，针对水印移除攻击的平均精度均值（mAP）绝对提升了19至29个点。