In-context learning (ICL) is an important capability of Large Language Models (LLMs), enabling these models to dynamically adapt based on specific, in-context exemplars, thereby improving accuracy and relevance. However, LLM's responses may leak the sensitive private information contained in in-context exemplars. To address this challenge, we propose Differentially Private In-context Learning (DP-ICL), a general paradigm for privatizing ICL tasks. The key idea for DP-ICL paradigm is generating differentially private responses through a noisy consensus among an ensemble of LLM's responses based on disjoint exemplar sets. Based on the general paradigm of DP-ICL, we instantiate several techniques showing how to privatize ICL for text classification and language generation. We evaluate DP-ICL on four text classification benchmarks and two language generation tasks, and our empirical results show that DP-ICL achieves a strong utility-privacy tradeoff.

翻译：上下文学习是大语言模型的一项重要能力，使模型能够根据具体的上下文示例动态调整，从而提高准确性和相关性。然而，大语言模型的响应可能泄露上下文示例中包含的敏感隐私信息。为解决这一挑战，我们提出差分隐私上下文学习，一种用于对上下文学习任务进行隐私化处理的一般范式。DP-ICL范式的核心思想是基于不相交示例集，通过集成大语言模型响应的噪声共识来生成差分隐私响应。基于DP-ICL的一般范式，我们实例化了多种技术，展示了如何对文本分类和语言生成的上下文学习进行隐私化处理。我们在四个文本分类基准和两个语言生成任务上对DP-ICL进行了评估，实证结果表明DP-ICL实现了良好的效用-隐私权衡。