Prompt recovery in large language models (LLMs) is crucial for understanding how LLMs work and addressing concerns regarding privacy, copyright, etc. The trend towards inference-only APIs complicates this task by restricting access to essential outputs for recovery. To tackle this challenge, we extract prompt-related information from limited outputs and identify a strong(negative) correlation between output probability-based uncertainty and the success of prompt recovery. This finding led to the development of Deliberative PrOmpt RecoverY (DORY), our novel approach that leverages uncertainty to recover prompts accurately. DORY involves reconstructing drafts from outputs, refining these with hints, and filtering out noise based on uncertainty. Our evaluation across diverse LLMs and prompt benchmarks shows that DORY outperforms existing baselines, improving performance by approximately 10.82% and establishing a new state-of-the-art record in prompt recovery tasks. Significantly, DORY operates using a single LLM without any external resources or model, offering a cost-effective, user-friendly prompt recovery solution.

翻译：在大语言模型（LLM）中，提示恢复对于理解LLM的工作原理以及解决隐私、版权等问题至关重要。仅提供推理功能的API趋势限制了恢复所需的关键输出访问，使得该任务变得更为复杂。为应对这一挑战，我们从有限的输出中提取与提示相关的信息，并发现基于输出概率的不确定性与提示恢复成功率之间存在显著的负相关关系。这一发现促使我们开发了审慎提示恢复方法（Deliberative PrOmpt RecoverY, DORY），这是一种利用不确定性来准确恢复提示的新方法。DORY包括从输出中重构草稿、利用线索优化这些草稿，以及基于不确定性过滤噪声。我们在多种LLM和提示基准测试上的评估表明，DORY优于现有基线方法，性能提升约10.82%，并在提示恢复任务中创造了新的最优记录。值得注意的是，DORY仅使用单一LLM运行，无需任何外部资源或模型，提供了一种经济高效、用户友好的提示恢复解决方案。