With the application of vertical domain pre-trained language models (VPLMs) in specialized fields such as medical, finance, and law, model parameters and inference capabilities have become important digital assets. Achieving traceable copyright verification for VPLMs has become an urgent challenge. Existing copyright verification methods primarily rely on embedding backdoor watermarks into models. However, most of these methods require additional training, suffer from inefficient watermark embedding, and lack scalable designs for multiple vertical domains. To address these limitations, we propose VertMark, the first unified training-free and robust watermarking framework for copyright verification across multiple vertical domain VPLMs. The framework embeds ownership-encoded watermarks by establishing a hidden semantic equivalence between low-frequency trigger tokens and high-frequency domain-relevant words via a training-free parameter replacement strategy. Experiments demonstrate that VertMark can achieve efficient watermark embedding and reliable watermark verification for both text understanding and text generation downstream tasks in the medical, financial, and legal domains, with negligible impact on model performance. Moreover, VertMark exhibits strong robustness against various attacks (e.g., pruning and quantization), highlighting its practical value and providing strong protection for the copyright security of VPLMs.

翻译：随着垂直领域预训练语言模型（VPLMs）在医疗、金融和法律等专业领域的应用，模型参数及推理能力已成为重要的数字资产。实现VPLMs可溯源的版权验证已成为迫切挑战。现有版权验证方法主要依赖将后门水印嵌入到模型中，但大多需要额外训练、存在水印嵌入效率低的问题，且缺乏面向多垂直领域的可扩展设计。针对这些局限，我们提出VertMark——首个面向多垂直领域VPLMs版权验证的统一免训练鲁棒水印框架。该框架通过一种免训练的参数替换策略，在低频触发令牌与高频领域相关词之间建立隐语义等价，从而嵌入所有权编码水印。实验表明，VertMark在医疗、金融和法律领域的文本理解与文本生成下游任务中，均能实现高效的水印嵌入及可靠的版权验证，且对模型性能影响可忽略。此外，VertMark对各类攻击（如剪枝与量化）展现出极强的鲁棒性，凸显其实用价值，为VPLMs的版权安全提供有力保护。