The recent explosion in the capabilities of large language models has led to a wave of interest in how best to prompt a model to perform a given task. While it may be tempting to simply choose a prompt based on average performance on a validation set, this can lead to a deployment where unexpectedly poor responses are generated, especially for the worst-off users. To mitigate this prospect, we propose Prompt Risk Control, a lightweight framework for selecting a prompt based on rigorous upper bounds on families of informative risk measures. We offer methods for producing bounds on a diverse set of metrics, including quantities that measure worst-case responses and disparities in generation quality across the population of users. In addition, we extend the underlying statistical bounding techniques to accommodate the possibility of distribution shifts in deployment. Experiments on applications such as open-ended chat, medical question summarization, and code generation highlight how such a framework can foster responsible deployment by reducing the risk of the worst outcomes.

翻译：近年来大语言模型能力的激增引发了对如何最佳地提示模型执行给定任务的广泛兴趣。虽然仅仅根据验证集上的平均性能选择提示可能颇具吸引力，但这可能导致部署中出现意外的不良响应，尤其是对最弱势的用户群体。为降低这种风险，我们提出提示风险控制——一个轻量级框架，通过基于信息性风险度量族严格上界的提示选择方法。我们针对多样化指标提供界限生成方法，包括衡量最差情形响应的指标及用户群体间生成质量差异的指标。此外，我们将底层统计界限技术扩展至部署时可能出现分布偏移的情况。在开放式对话、医学问题摘要和代码生成等应用上的实验表明，该框架通过降低最差结果的风险可促进负责任的部署。