Combining predictions from different models is a central problem in Bayesian inference and machine learning more broadly. Currently, these predictive distributions are almost exclusively combined using linear mixtures such as Bayesian model averaging, Bayesian stacking, and mixture of experts. Such linear mixtures impose idiosyncrasies that might be undesirable for some applications, such as multi-modality. While there exist alternative strategies (e.g. geometric bridge or superposition), optimising their parameters usually involves computing an intractable normalising constant repeatedly. We present two novel Bayesian model combination tools. These are generalisations of model stacking, but combine posterior densities by log-linear pooling (locking) and quantum superposition (quacking). To optimise model weights while avoiding the burden of normalising constants, we investigate the Hyvarinen score of the combined posterior predictions. We demonstrate locking with an illustrative example and discuss its practical application with importance sampling.

翻译：将不同模型的预测结果进行组合是贝叶斯推断及更广泛的机器学习中的核心问题。目前，这些预测分布几乎完全通过线性混合方式（如贝叶斯模型平均、贝叶斯堆叠和专家混合）进行组合。此类线性混合会引入某些应用中可能不理想的特性（如多模态性）。尽管存在替代策略（例如几何桥接或量子叠加），但优化其参数通常需要反复计算难以处理的归一化常数。我们提出了两种新颖的贝叶斯模型组合工具。这些工具是模型堆叠的泛化形式，但通过对数线性池化（锁定）和量子叠加（扑腾）来组合后验密度。为在优化模型权重时避免归一化常数的计算负担，我们研究了组合后验预测的Hyvarinen评分。我们通过一个说明性示例演示了锁定方法，并讨论了其在重要性采样中的实际应用。