We introduce the framework of "social learning" in the context of large language models (LLMs), whereby models share knowledge with each other in a privacy-aware manner using natural language. We present and evaluate two approaches for knowledge transfer between LLMs. In the first scenario, we allow the model to generate abstract prompts aiming to teach the task. In our second approach, models transfer knowledge by generating synthetic examples. We evaluate these methods across diverse datasets and quantify memorization as a proxy for privacy loss. These techniques inspired by social learning yield promising results with low memorization of the original data. In particular, we show that performance using these methods is comparable to results with the use of original labels and prompts. Our work demonstrates the viability of social learning for LLMs, establishes baseline approaches and highlights several unexplored areas for future work.

翻译：我们提出了大语言模型（LLMs）背景下“社会学习”的框架，其中模型以隐私保护的方式通过自然语言相互共享知识。我们提出并评估了两种在LLMs之间进行知识迁移的方法。第一种场景中，我们允许模型生成旨在教授任务的抽象提示。第二种方法中，模型通过生成合成样例来迁移知识。我们在不同数据集上评估了这些方法，并以记忆化作为隐私损失的代理指标进行量化。受社会学习启发的这些技术产生了有前景的结果，且原始数据的记忆化程度较低。特别地，我们表明使用这些方法的性能与使用原始标签和提示的结果相当。我们的工作展示了社会学习在LLMs中的可行性，建立了基线方法，并突出了未来工作中若干未探索的领域。