Large language models (LLMs) have demonstrated remarkable capabilities out of box for a wide range of applications, yet accuracy still remains a major growth area, especially in mission-critical domains such as biomedicine. An effective method to calibrate the confidence level on LLM responses is essential to automatically detect errors and facilitate human-in-the-loop verification. An important source of calibration signals stems from expert-stipulated programmatic supervision, which is often available at low cost but has its own limitations such as noise and coverage. In this paper, we introduce a Pareto optimal self-supervision framework that can leverage available programmatic supervision to systematically calibrate LLM responses by producing a risk score for every response, without any additional manual efforts. This is accomplished by learning a harmonizer model to align LLM output with other available supervision sources, which would assign higher risk scores to more uncertain LLM responses and facilitate error correction. Experiments on standard relation extraction tasks in biomedical and general domains demonstrate the promise of this approach, with our proposed risk scores highly correlated with the real error rate of LLMs. For the most uncertain test instances, dynamic prompting based on our proposed risk scores results in significant accuracy improvement for off-the-shelf LLMs, boosting GPT-3 results past state-of-the-art (SOTA) weak supervision and GPT-4 results past SOTA supervised results on challenging evaluation datasets.

翻译：大型语言模型（LLM）在广泛应用中展现出开箱即用的卓越能力，但其准确性仍是关键提升领域，尤其是在生物医学等关键任务领域。有效校准LLM响应置信水平的校准方法对于自动检测错误和促进人在回路验证至关重要。校准信号的重要来源之一是专家制定的程序化监督，这类监督通常成本较低，但存在噪声和覆盖范围有限等固有局限。本文提出一种帕累托最优自监督框架，该框架可利用现有程序化监督，通过为每个响应生成风险评分来系统地校准LLM响应，无需额外人工干预。通过学习一个协调器模型，使LLM输出与其他可用监督源对齐，为更不确定的LLM响应分配更高风险评分，从而促进误差校正。在生物医学和通用领域标准关系抽取任务上的实验证明了该方法的潜力，我们提出的风险评分与LLM实际错误率高度相关。对于最不确定的测试实例，基于所提风险评分的动态提示可使现成LLM的准确性显著提升：将GPT-3的结果提升至超越最先进弱监督方法，并将GPT-4的结果提升至在具有挑战性的评估数据集上超越最先进监督方法。