Large language models (LLMs) are excellent in-context learners. However, the sensitivity of data contained in prompts raises privacy concerns. Our work first shows that these concerns are valid: we instantiate a simple but highly effective membership inference attack against the data used to prompt LLMs. To address this vulnerability, one could forego prompting and resort to fine-tuning LLMs with known algorithms for private gradient descent. However, this comes at the expense of the practicality and efficiency offered by prompting. Therefore, we propose to privately learn to prompt. We first show that soft prompts can be obtained privately through gradient descent on downstream data. However, this is not the case for discrete prompts. Thus, we orchestrate a noisy vote among an ensemble of LLMs presented with different prompts, i.e., a flock of stochastic parrots. The vote privately transfers the flock's knowledge into a single public prompt. We show that LLMs prompted with our private algorithms closely match the non-private baselines. For example, using GPT3 as the base model, we achieve a downstream accuracy of 92.7% on the sst2 dataset with ($\epsilon=0.147, \delta=10^{-6}$)-differential privacy vs. 95.2% for the non-private baseline. Through our experiments, we also show that our prompt-based approach is easily deployed with existing commercial APIs.

翻译：大语言模型（LLMs）是优秀的上下文学习器。然而，提示中包含的数据敏感性引发了隐私担忧。我们的工作首先证实了这些担忧的合理性：我们针对用于提示LLMs的数据，实例化了一种简单但高效的成员推断攻击。为解决此漏洞，一种方法是放弃提示学习，转而采用已知算法对LLMs进行私有梯度下降微调。但这会牺牲提示学习带来的实用性和效率。因此，我们提出私有化学习提示。我们首先证明了软提示可通过下游数据上的梯度下降实现私有化。但离散提示则不然。为此，我们协调一个由不同提示构成的LLMs集成（即随机鹦鹉群）进行噪声投票，并通过投票将集成知识私有化地迁移到单个公开提示中。实验表明，使用我们的私有化算法提示的LLMs能紧密匹配非私有基线。例如，以GPT3为基础模型，我们在sst2数据集上实现了92.7%的下游准确率（满足($\epsilon=0.147, \delta=10^{-6}$)-差分隐私），而非私有基线为95.2%。通过实验，我们还证明了基于提示的方法可轻松部署于现有商业API。