As Large Language Models (LLMs) gain great success in real-world applications, an increasing number of users are seeking to develop and deploy their customized LLMs through cloud services. Nonetheless, in some specific domains, there are still concerns regarding cost and trade-offs between privacy issues and accuracy. In this study, we introduce a cost-effective and self-adaptive LLM shaking tuning and recovery mechanism, named CypherTalk. With carefully designed horizontal and vertical shaking operators, we can achieve comparable accuracy results with SOTA privacy-preserving LLM schemes using Cryptography-based or Differential Privacy-based methods. Experiments also show that with the CypherTalk framework, users can achieve reliable accuracy when using optimized shaking operator settings. To our best knowledge, this is the first work that considers cost, and trade-off between model utility and privacy in LLM scenarios.

翻译：随着大型语言模型（LLM）在实际应用中取得巨大成功，越来越多的用户寻求通过云服务开发并部署其定制化LLM。然而，在特定领域中，成本问题以及隐私与准确性之间的权衡仍存在隐忧。本研究提出一种名为CypherTalk的经济高效且自适应的LLM抖动调优与恢复机制。通过精心设计的横向与纵向抖动算子，我们能够在使用基于密码学或差分隐私方法的现有隐私保护LLM方案时，达到可比的准确性结果。实验表明，采用CypherTalk框架，用户可在优化抖动算子设置下获得可靠的准确性。据我们所知，这是首次在LLM场景中综合考虑成本及模型效用与隐私之间权衡的工作。