Offsite-tuning is a privacy-preserving method for tuning large language models (LLMs) by sharing a lossy compressed emulator from the LLM owners with data owners for downstream task tuning. This approach protects the privacy of both the model and data owners. However, current offsite tuning methods often suffer from adaptation degradation, high computational costs, and limited protection strength due to uniformly dropping LLM layers or relying on expensive knowledge distillation. To address these issues, we propose ScaleOT, a novel privacy-utility-scalable offsite-tuning framework that effectively balances privacy and utility. ScaleOT introduces a novel layerwise lossy compression algorithm that uses reinforcement learning to obtain the importance of each layer. It employs lightweight networks, termed harmonizers, to replace the raw LLM layers. By combining important original LLM layers and harmonizers in different ratios, ScaleOT generates emulators tailored for optimal performance with various model scales for enhanced privacy protection. Additionally, we present a rank reduction method to further compress the original LLM layers, significantly enhancing privacy with negligible impact on utility. Comprehensive experiments show that ScaleOT can achieve nearly lossless offsite tuning performance compared with full fine-tuning while obtaining better model privacy.

翻译：离线调优是一种保护隐私的大型语言模型调优方法，通过将大型语言模型所有者提供的损失性压缩仿真器与数据所有者共享，用于下游任务调优。该方法同时保护了模型所有者和数据所有者的隐私。然而，现有离线调优方法常因均匀丢弃大型语言模型层或依赖昂贵的知识蒸馏而导致适应性能下降、计算成本高昂且保护强度有限。为解决这些问题，我们提出ScaleOT——一种新颖的隐私-效用可扩展离线调优框架，能有效平衡隐私与效用。ScaleOT引入了一种创新的分层损失性压缩算法，利用强化学习获取各层重要性。该框架采用轻量级网络（称为协调器）替换原始大型语言模型层。通过以不同比例组合重要的原始大型语言模型层与协调器，ScaleOT可生成针对不同模型规模优化的仿真器，从而增强隐私保护。此外，我们提出一种秩约简方法以进一步压缩原始大型语言模型层，在几乎不影响效用的前提下显著提升隐私性。综合实验表明，与全参数微调相比，ScaleOT能实现近乎无损的离线调优性能，同时获得更优的模型隐私保护效果。