Temperature scaling is a simple method that allows to control the uncertainty of probabilistic models. It is mostly used in two contexts: improving the calibration of classifiers and tuning the stochasticity of large language models (LLMs). In both cases, temperature scaling is the most popular method for the job. Despite its popularity, a rigorous theoretical analysis of the properties of temperature scaling has remained elusive. We investigate here some of these properties. For classification, we show that increasing the temperature increases the uncertainty in the model in a very general sense (and in particular increases its entropy). However, for LLMs, we challenge the common claim that increasing temperature increases diversity. Furthermore, we introduce two new characterisations of temperature scaling. The first one is geometric: the tempered model is shown to be the information projection of the original model onto the set of models with a given entropy. The second characterisation clarifies the role of temperature scaling as a submodel of more general linear scalers such as matrix scaling and Dirichlet calibration: we show that temperature scaling is the only linear scaler that does not change the hard predictions of the model.

翻译：温度缩放是一种控制概率模型不确定性的简单方法。它主要应用于两种场景：改善分类器的校准度，以及调节大型语言模型（LLMs）的随机性。在这两种情况下，温度缩放都是最常用的方法。尽管其应用广泛，但关于温度缩放性质的严格理论分析仍较为缺乏。本文研究了该方法的若干性质。对于分类任务，我们证明提高温度会在非常普遍的意义上增加模型的不确定性（特别是会增加其熵）。然而，对于LLMs，我们质疑了“提高温度会增加多样性”这一普遍观点。此外，我们提出了温度缩放的两种新表征。第一种是几何表征：温度缩放后的模型可视为原始模型在给定熵约束下的信息投影。第二种表征阐明了温度缩放作为更广义线性缩放器（如矩阵缩放和狄利克雷校准）子模型的作用：我们证明温度缩放是唯一不改变模型硬预测结果的线性缩放器。