Watermarking has emerged as a promising technique for tracing the authorship of content generated by large language models (LLMs). Among existing approaches, the KGW scheme is particularly attractive due to its versatility, efficiency, and effectiveness in natural language generation. However, KGW's effectiveness degrades significantly under low-entropy settings such as code generation and mathematical reasoning. A crucial step in the KGW method is random vocabulary partitioning, which enables adjustments to token selection based on specific preferences. Our study revealed that the next-token probability distribution plays an critical role in determining how much, or even whether, we can modify token selection and, consequently, the effectiveness of watermarking. We refer to this characteristic, associated with the probability distribution of each token prediction, as \emph{watermark strength.} In cases of random vocabulary partitioning, the lower bound of watermark strength is dictated by the next-token probability distribution. However, we found that, by redesigning the vocabulary partitioning algorithm, we can potentially raise this lower bound. In this paper, we propose SSG (\textbf{S}ort-then-\textbf{S}plit by \textbf{G}roups), a method that partitions the vocabulary into two logit-balanced subsets. This design lifts the lower bound of watermark strength for each token prediction, thereby improving watermark detectability. Experiments on code generation and mathematical reasoning datasets demonstrate the effectiveness of SSG.

翻译：水印技术已成为追踪大语言模型生成内容溯源的一种有效手段。在现有方法中，KGW方案因其在自然语言生成中的通用性、高效性和有效性而备受关注。然而，在代码生成和数学推理等低熵场景下，KGW的性能会显著下降。KGW方法的关键步骤是随机词汇划分，该步骤能够基于特定偏好调整token选择。我们的研究发现，下一个token概率分布在决定token选择的可调整程度（甚至能否调整）以及水印有效性方面起着关键作用。我们将与每个token预测概率分布相关的特性称为"水印强度"。在随机词汇划分的情况下，水印强度的下限由下一个token概率分布决定。但我们发现，通过重新设计词汇划分算法，可以提升该下限。本文提出SSG（分组排序分割）方法，将词汇表划分为两个logit平衡的子集。该设计提升了每个token预测的水印强度下限，从而增强了水印的可检测性。在代码生成和数学推理数据集上的实验验证了SSG方法的有效性。