Temperature scaling is a simple method that allows to control the uncertainty of probabilistic models. It is mostly used in two contexts: improving the calibration of classifiers and tuning the stochasticity of large language models (LLMs). In both cases, temperature scaling is the most popular method for the job. Despite its popularity, a rigorous theoretical analysis of the properties of temperature scaling has remained elusive. We investigate here some of these properties. For classification, we show that increasing the temperature increases the uncertainty in the model in a very general sense (and in particular increases its entropy). However, for LLMs, we challenge the common claim that increasing temperature increases diversity. Furthermore, we introduce two new characterisations of temperature scaling. The first one is geometric: the tempered model is shown to be the information projection of the original model onto the set of models with a given entropy. The second characterisation clarifies the role of temperature scaling as a submodel of more general linear scalers such as matrix scaling and Dirichlet calibration: we show that temperature scaling is the only linear scaler that does not change the hard predictions of the model.

翻译：温度缩放是一种能够控制概率模型不确定性的简单方法。它主要用于两种场景：改进分类器的校准效果，以及调节大型语言模型（LLM）的随机性。在这两种场景中，温度缩放都是最常用的方法。尽管应用广泛，但对温度缩放性质的严格理论分析却一直缺失。本文研究其中若干性质。对于分类任务，我们证明提高温度会在非常一般的意义上增加模型的不确定性（尤其会提高其熵值）。然而，对于LLM，我们质疑了“提高温度会提升多样性”的常见论断。此外，我们提出温度缩放的两种新表征。第一种是几何表征：经过温度调节的模型被证明是原始模型在具有给定熵值的模型集合上的信息投影。第二种表征阐明了温度缩放作为更广义线性缩放器（如矩阵缩放和狄利克雷校准）子模型的作用：我们证明温度缩放是唯一不改变模型硬预测结果的线性缩放器。