Villalobos et al. [2024] predict that publicly available human text will be exhausted within the next decade. Thus, improving models without access to ground-truth labels becomes increasingly important. We propose a label-free post-processing framework that improves a strong but miscalibrated model using a weaker yet better-calibrated reference. Our framework guarantees a strict performance improvement under any proper loss. Our approach is based on a characterization of when strict improvement is possible: when the strong and reference models are not mutually calibrated. We formalize this condition, connect it to arbitrage and no-trade results from economics, and develop an efficient Bregman projection algorithm that guarantees worst-case loss reduction without labels. Experiments on representative LLMs across varying scales demonstrate that our label-free method significantly reduces proper losses and calibration errors, achieving performance competitive with supervised baselines.

翻译：Villalobos等人[2024]预测，公开可用的人类文本将在未来十年内耗尽。因此，在无法获取真实标签的情况下改进模型变得日益重要。我们提出了一种无需标签的后处理框架，该框架利用一个较弱但校准更优的参考模型来改进强大但校准不佳的模型。我们的框架保证在任何严格损失函数下都能实现严格的性能提升。该方法基于对严格改进可能性的理论刻画：当强模型与参考模型未达到相互校准状态时。我们形式化了这一条件，将其与经济学中的套利及无交易定理相联系，并开发了一种高效的布雷格曼投影算法，该算法能在无标签情况下保证最坏情况下的损失降低。在不同规模的典型大语言模型上的实验表明，我们的无监督方法能显著降低严格损失与校准误差，其性能可与有监督基线方法相媲美。