Language generation models have been an increasingly powerful enabler for many applications. Many such models offer free or affordable API access, which makes them potentially vulnerable to model extraction attacks through distillation. To protect intellectual property (IP) and ensure fair use of these models, various techniques such as lexical watermarking and synonym replacement have been proposed. However, these methods can be nullified by obvious countermeasures such as "synonym randomization". To address this issue, we propose GINSEW, a novel method to protect text generation models from being stolen through distillation. The key idea of our method is to inject secret signals into the probability vector of the decoding steps for each target token. We can then detect the secret message by probing a suspect model to tell if it is distilled from the protected one. Experimental results show that GINSEW can effectively identify instances of IP infringement with minimal impact on the generation quality of protected APIs. Our method demonstrates an absolute improvement of 19 to 29 points on mean average precision (mAP) in detecting suspects compared to previous methods against watermark removal attacks.

翻译：语言生成模型正日益成为众多应用领域的有力推动者。许多此类模型提供免费或低成本的API访问，这使其可能通过蒸馏技术遭受模型提取攻击。为保护知识产权（IP）并确保模型的合理使用，研究者已提出多种技术，如词汇水印和同义词替换。然而，这些方法可能被诸如“同义词随机化”等明显的对抗措施所抵消。针对这一问题，我们提出GINSEW，一种保护文本生成模型免遭蒸馏窃取的新方法。该方法的核心思想是，在解码步骤中为每个目标词元的概率向量注入秘密信号。随后，通过探查可疑模型，我们可以检测其是否源自受保护模型的蒸馏，从而判断是否存在秘密信号。实验结果表明，GINSEW能够在最小化对受保护API生成质量影响的前提下，有效识别知识产权侵权行为。与以往方法相比，我们的方法在对抗水印移除攻击时的平均检测精度（mAP）上实现了19至29个百分点的绝对提升。