When users submit queries to Large Language Models (LLMs), their prompts can often contain sensitive data, forcing a difficult choice: Send the query to a powerful proprietary LLM providers to achieving state-of-the-art performance and risk data exposure, or relying on smaller, local models guarantees data privacy but often results in a degradation of task performance. Prior approaches have relied on static pipelines that use LLM rewriting, which shatters linguistic coherence and indiscriminately removes privacy-sensitive information, including task-critical content. We reformulate this challenge (Privacy-Conscious Delegation) as a sequential decision-making problem and introduce a novel reinforcement learning (RL) framework called Privacy-R1 to solve it. Our framework trains an agent to dynamically route text chunks, learning a policy that optimally balances the trade-off between privacy leakage and task performance. It implicitly distinguishes between replaceable Personally Identifiable Information (PII) (which it shields locally) and task-critical PII (which it strategically sends to the remote model for maximal utility). To validate our approach in complex scenarios, we also introduce a new medical dataset with high PII density. Our framework achieves a new state-of-the-art on the privacy-utility frontier, demonstrating the necessity of learned, adaptive policies for deploying LLMs in sensitive environments. Dataset can be found at: https://github.com/zackhuiiiii/Privacy-R1.

翻译：用户向大型语言模型（LLMs）提交查询时，其提示词常包含敏感数据，迫使面临两难抉择：将查询发送至强大的专有LLM提供商以获得最先进性能但面临数据泄露风险，或依赖本地小模型保障数据隐私却导致任务性能下降。现有方法依赖静态流水线，通过LLM重写文本，这既破坏了语言连贯性，又无差别移除包括任务关键内容在内的隐私敏感信息。我们将这一挑战（隐私感知委托）重构为序列决策问题，并提出名为Privacy-R1的新型强化学习框架加以解决。该框架训练智能体动态路由文本分块，学习能优化隐私泄露与任务性能平衡的策略。它隐式区分可替换的个人身份信息（由本地屏蔽）与任务关键型PII（策略性发送至远程模型以最大化效用）。为在复杂场景验证方法，我们还引入高PII密度的新型医学数据集。本框架在隐私-效用前沿达到新最先进水平，论证了在敏感环境中部署LLM时采用学习型自适应策略的必要性。数据集地址：https://github.com/zackhuiiiii/Privacy-R1