Large language models (LLMs) show early signs of artificial general intelligence but struggle with hallucinations. One promising solution to mitigate these hallucinations is to store external knowledge as embeddings, aiding LLMs in retrieval-augmented generation. However, such a solution risks compromising privacy, as recent studies experimentally showed that the original text can be partially reconstructed from text embeddings by pre-trained language models. The significant advantage of LLMs over traditional pre-trained models may exacerbate these concerns. To this end, we investigate the effectiveness of reconstructing original knowledge and predicting entity attributes from these embeddings when LLMs are employed. Empirical findings indicate that LLMs significantly improve the accuracy of two evaluated tasks over those from pre-trained models, regardless of whether the texts are in-distribution or out-of-distribution. This underscores a heightened potential for LLMs to jeopardize user privacy, highlighting the negative consequences of their widespread use. We further discuss preliminary strategies to mitigate this risk.

翻译：大型语言模型（LLMs）展现出人工通用智能的早期迹象，但在处理幻觉现象方面仍存在困难。为缓解此类幻觉，一种有前景的方案是将外部知识以嵌入形式存储，辅助LLMs进行检索增强生成。然而，这种方案存在隐私泄露的风险——近期实验研究表明，预训练语言模型可从文本嵌入中部分重建原始文本。相较于传统预训练模型，LLMs的显著优势可能加剧这种担忧。为此，我们探究了在采用LLMs时，从嵌入中重建原始知识及预测实体属性的有效性。实证结果表明，无论文本是否属于分布内或分布外，LLMs在两个评估任务中的准确性均较预训练模型有显著提升。这凸显了LLMs损害用户隐私的更大可能性，揭示了其广泛应用所带来的负面后果。我们进一步讨论了缓解该风险的初步策略。