System Instructions in Large Language Models (LLMs) are commonly used to enforce safety policies, define agent behavior, and protect sensitive operational context in agentic AI applications. These instructions may contain sensitive information such as API credentials, internal policies, and privileged workflow definitions, making system instruction leakage a critical security risk highlighted in the OWASP Top 10 for LLM Applications. Without incurring the overhead costs of reasoning models, many LLM applications rely on refusal-based instructions that block direct requests for system instructions, implicitly assuming that prohibited information can only be extracted through explicit queries. We introduce an automated evaluation framework that tests whether system instructions remain confidential when extraction requests are re-framed as encoding or structured output tasks. Across four common models and 46 verified system instructions, we observe high attack success rates (> 0.7) for structured serialization where models refuse direct extraction requests but disclose protected content in the requested serialization formats. We further demonstrate a mitigation strategy based on one-shot instruction reshaping using a Chain-of-Thought reasoning model, indicating that even subtle changes in wording and structure of system instructions can significantly reduce attack success rate without requiring model retraining.

翻译：大语言模型（LLM）中的系统指令常用于强制执行安全策略、定义代理行为，以及保护智能体AI应用中敏感的操作上下文。这些指令可能包含API凭证、内部策略和特权工作流定义等敏感信息，使得系统指令泄露成为OWASP LLM应用十大安全风险中强调的关键安全问题。在不承担推理模型额外计算成本的前提下，许多LLM应用依赖基于拒绝的指令来拦截对系统指令的直接请求，隐含假设禁止信息只能通过显式查询被提取。我们提出一个自动化评估框架，测试当提取请求被重新框架为编码或结构化输出任务时，系统指令是否仍能保持机密性。在四种常见模型和46个经过验证的系统指令中，针对结构化序列化攻击（即模型拒绝直接提取请求，但以请求的序列化格式泄露受保护内容）均观察到高攻击成功率（>0.7）。我们进一步展示了一种基于单样本指令重塑的防御策略——利用思维链推理模型对系统指令进行微调，表明仅通过改变指令的措辞和结构，无需重新训练模型即可显著降低攻击成功率。