As large language models (LLMs) generate texts with increasing fluency and realism, there is a growing need to identify the source of texts to prevent the abuse of LLMs. Text watermarking techniques have proven reliable in distinguishing whether a text is generated by LLMs by injecting hidden patterns into the generated texts. However, we argue that existing watermarking methods for LLMs are encoding-inefficient (only contain one bit of information - whether it is generated from an LLM or not) and cannot flexibly meet the diverse information encoding needs (such as encoding model version, generation time, user id, etc.) in different LLMs application scenarios. In this work, we conduct the first systematic study on the topic of Codable Text Watermarking for LLMs (CTWL) that allows text watermarks to carry more customizable information. First of all, we study the taxonomy of LLM watermarking technology and give a mathematical formulation for CTWL. Additionally, we provide a comprehensive evaluation system for CTWL: (1) watermarking success rate, (2) robustness against various corruptions, (3) coding rate of payload information, (4) encoding and decoding efficiency, (5) impacts on the quality of the generated text. To meet the requirements of these non-Pareto-improving metrics, we devise a CTWL method named Balance-Marking, based on the motivation of ensuring that available and unavailable vocabularies for encoding information have approximately equivalent probabilities. Compared to the random vocabulary partitioning extended from the existing work, a probability-balanced vocabulary partition can significantly improve the quality of the generated text. Extensive experimental results have shown that our method outperforms a direct baseline under comprehensive evaluation.

翻译：随着大型语言模型（LLMs）生成文本的流畅度和逼真度不断提升，识别文本来源以防止LLMs滥用的需求日益迫切。文本水印技术通过在生成的文本中注入隐藏模式，已被证明能够可靠地区分文本是否由LLMs生成。然而，我们认为现有面向LLMs的水印方法存在编码效率低下的问题（仅包含单比特信息——即文本是否由LLM生成），无法灵活满足不同LLM应用场景中多样化的信息编码需求（例如编码模型版本、生成时间、用户ID等）。本研究首次系统性地探讨了面向LLMs的可编码文本水印（CTWL）这一课题，该技术使文本水印能够携带更多可定制化信息。首先，我们研究了LLM水印技术的分类体系，并给出了CTWL的数学形式化描述。此外，我们为CTWL构建了综合评价体系：（1）水印成功率，（2）对各种篡改的鲁棒性，（3）有效载荷的信息编码率，（4）编码与解码效率，（5）对生成文本质量的影响。为满足这些非帕累托改进型指标的要求，我们提出了一种名为Balance-Marking的CTWL方法，其核心思想是确保可用于编码信息的词汇与不可用词汇具有近似相等的概率。与现有研究中扩展的随机词汇划分相比，概率均衡的词汇划分能够显著提升生成文本的质量。大量实验结果表明，在全面评估下，我们的方法优于直接基线方法。