Deep search agents have emerged as a promising paradigm for addressing complex information-seeking tasks, but their training remains challenging due to sparse rewards, weak credit assignment, and limited labeled data. Self-play offers a scalable route to reduce data dependence, but conventional self-play optimizes students only through sparse outcome rewards, leading to low learning efficiency. In this work, we observe that self-play naturally produces a question construction path (QCP) during task generation, an intermediate artifact that captures the reverse solution process. This reveals a new source of privileged information for self-distillation: self-play can itself provide high-quality privileged context for the teacher model in a low-cost and scalable manner, without relying on human feedback or curated privileged information. Leveraging this insight, we propose Privileged Information Self-Play ($π$-Play), a multi-agent self-evolution framework. In $π$-Play, an examiner generates tasks together with their QCPs, and a teacher model leverages QCP as privileged context to densely supervise a student via self-distillation. This design transforms conventional sparse-reward self-play into a dense-feedback self-evolution loop. Extensive experiments show that data-free $π$-Play surpasses fully supervised search agents and improves evolutionary efficiency by 2-3$\times$ over conventional self-play.

翻译：深度搜索智能体已成为解决复杂信息获取任务的一种有前景范式，但其训练仍面临稀疏奖励、弱信用分配和标注数据有限等挑战。自博弈为减少数据依赖提供了一条可扩展的路径，但传统自博弈仅通过稀疏结果奖励优化学生模型，导致学习效率低下。本研究发现，自博弈在任务生成过程中自然会产生问题构建路径（QCP），这是一种捕获逆向求解过程的中间产物。这揭示了自蒸馏中一种全新的特权信息来源：自博弈本身能够以低成本、可扩展的方式为教师模型提供高质量特权上下文，而无需依赖人类反馈或精心设计的特权信息。基于这一洞察，我们提出特权信息自博弈（$π$-Play），一种多智能体自进化框架。在$π$-Play中，审查者生成任务及其QCP，教师模型利用QCP作为特权上下文，通过自蒸馏对学生模型进行密集监督。该设计将传统的稀疏奖励自博弈转化为密集反馈的自进化循环。大量实验表明，无外部数据的$π$-Play超越了完全监督的搜索智能体，并将进化效率较传统自博弈提升2-3倍。