Recent advances in vision-language pre-trained models (VLPs) have significantly increased visual understanding and cross-modal analysis capabilities. Companies have emerged to provide multi-modal Embedding as a Service (EaaS) based on VLPs (e.g., CLIP-based VLPs), which cost a large amount of training data and resources for high-performance service. However, existing studies indicate that EaaS is vulnerable to model extraction attacks that induce great loss for the owners of VLPs. Protecting the intellectual property and commercial ownership of VLPs is increasingly crucial yet challenging. A major solution of watermarking model for EaaS implants a backdoor in the model by inserting verifiable trigger embeddings into texts, but it is only applicable for large language models and is unrealistic due to data and model privacy. In this paper, we propose a safe and robust backdoor-based embedding watermarking method for VLPs called VLPMarker. VLPMarker utilizes embedding orthogonal transformation to effectively inject triggers into the VLPs without interfering with the model parameters, which achieves high-quality copyright verification and minimal impact on model performance. To enhance the watermark robustness, we further propose a collaborative copyright verification strategy based on both backdoor trigger and embedding distribution, enhancing resilience against various attacks. We increase the watermark practicality via an out-of-distribution trigger selection approach, removing access to the model training data and thus making it possible for many real-world scenarios. Our extensive experiments on various datasets indicate that the proposed watermarking approach is effective and safe for verifying the copyright of VLPs for multi-modal EaaS and robust against model extraction attacks. Our code is available at https://github.com/Pter61/vlpmarker.

翻译：近期视觉-语言预训练模型（VLPs）的进展显著提升了视觉理解与跨模态分析能力。基于VLPs（如CLIP类模型）的多模态嵌入即服务（EaaS）已投入商业应用，但其高性能服务需要消耗大量训练数据和计算资源。然而现有研究表明，EaaS易受模型窃取攻击，导致VLP所有者蒙受重大损失。保护VLPs的知识产权与商业所有权日益重要且充满挑战。主流的水印方案通过向文本插入可验证触发器植入后门，但仅适用于大语言模型且因数据与模型隐私限制而难以实际部署。本文提出面向VLPs的安全鲁棒后门嵌入水印方法VLPMarker，该方法通过嵌入正交变换在不干扰模型参数的前提下高效注入触发器，实现高质量版权验证且对模型性能影响极小。为增强水印鲁棒性，我们进一步提出基于后门触发与嵌入分布的协同版权验证策略，提升抵御各类攻击的能力。通过采用分布外触发器选择方法提高水印实用性，无需访问模型训练数据即可适配多种真实场景。在多个数据集上的大量实验表明，所提水印方法能有效且安全地验证多模态EaaS中VLPs的版权，并对模型窃取攻击具备鲁棒性。我们的代码开源至https://github.com/Pter61/vlpmarker。