As large language models (LLMs) become integrated into sensitive workflows, concerns grow over their potential to leak confidential information. We propose TrojanStego, a novel threat model in which an adversary fine-tunes an LLM to embed sensitive context information into natural-looking outputs via linguistic steganography, without requiring explicit control over inference inputs. We introduce a taxonomy outlining risk factors for compromised LLMs, and use it to evaluate the risk profile of the threat. To implement TrojanStego, we propose a practical encoding scheme based on vocabulary partitioning learnable by LLMs via fine-tuning. Experimental results show that compromised models reliably transmit 32-bit secrets with 87% accuracy on held-out prompts, reaching over 97% accuracy using majority voting across three generations. Further, they maintain high utility, can evade human detection, and preserve coherence. These results highlight a new class of LLM data exfiltration attacks that are passive, covert, practical, and dangerous.

翻译：随着大型语言模型（LLMs）被集成到敏感工作流程中，人们对其可能泄露机密信息的担忧日益增长。我们提出特洛伊隐写（TrojanStego），一种新颖的威胁模型：攻击者通过微调LLM，利用语言隐写术将敏感上下文信息嵌入到看似自然的输出中，而无需显式控制推理输入。我们引入了一个分类法，概述了受损LLM的风险因素，并利用该分类法评估了此威胁的风险特征。为实现特洛伊隐写，我们提出了一种基于词汇划分的实用编码方案，该方案可通过微调由LLMs学习。实验结果表明，受损模型能够在保留提示上以87%的准确率可靠传输32位秘密，通过三次生成中的多数投票，准确率可超过97%。此外，这些模型保持了高实用性，能够逃避人工检测，并保持连贯性。这些结果突显了一类新型的LLM数据外泄攻击，其具有被动性、隐蔽性、实用性和危险性。