The generative Artificial Intelligence (AI) tools based on Large Language Models (LLMs) use billions of parameters to extensively analyse large datasets and extract critical private information such as, context, specific details, identifying information etc. This have raised serious threats to user privacy and reluctance to use such tools. This article proposes the conceptual model called PrivChatGPT, a privacy-preserving model for LLMs that consists of two main components i.e., preserving user privacy during the data curation/pre-processing together with preserving private context and the private training process for large-scale data. To demonstrate its applicability, we show how a private mechanism could be integrated into the existing model for training LLMs to protect user privacy; specifically, we employed differential privacy and private training using Reinforcement Learning (RL). We measure the privacy loss and evaluate the measure of uncertainty or randomness once differential privacy is applied. It further recursively evaluates the level of privacy guarantees and the measure of uncertainty of public database and resources, during each update when new information is added for training purposes. To critically evaluate the use of differential privacy for private LLMs, we hypothetically compared other mechanisms e..g, Blockchain, private information retrieval, randomisation, for various performance measures such as the model performance and accuracy, computational complexity, privacy vs. utility etc. We conclude that differential privacy, randomisation, and obfuscation can impact utility and performance of trained models, conversely, the use of ToR, Blockchain, and PIR may introduce additional computational complexity and high training latency. We believe that the proposed model could be used as a benchmark for proposing privacy preserving LLMs for generative AI tools.

翻译：基于大型语言模型的生成式人工智能工具利用数十亿个参数对大规模数据集进行深度分析，并提取上下文、具体细节、身份信息等关键隐私数据。这引发了用户隐私的严重威胁，并导致用户对此类工具的使用产生顾虑。本文提出名为PrivChatGPT的概念模型，这是一种面向大型语言模型的隐私保护框架，由两大核心组件构成：在数据整理/预处理阶段保护用户隐私，以及在规模化数据训练过程中保障私有上下文与训练流程的隐私性。为验证其可行性，我们展示了如何将隐私保护机制集成到现有的大型语言模型训练框架中，具体采用差分隐私技术结合基于强化学习的隐私训练方法。通过测量应用差分隐私后的隐私损失，我们量化了不确定性或随机性指标。该方法进一步递归评估了在每次训练数据更新时，公共数据库与资源所对应的隐私保证水平及不确定性程度。为批判性评估差分隐私在隐私保护型大型语言模型中的应用效果，我们假设性对比了区块链、私有信息检索、随机化等机制在模型性能、准确率、计算复杂度、隐私与效用权衡等多项指标上的表现。研究结论表明：差分隐私、随机化与混淆技术可能影响训练模型的效用与性能，而使用洋葱路由、区块链及私有信息检索则可能引入额外计算复杂度并导致训练延迟增加。我们相信，所提出的模型可作为生成式AI工具中隐私保护型大型语言模型的基准参考。