A Bayesian pseudocoreset is a compact synthetic dataset summarizing essential information of a large-scale dataset and thus can be used as a proxy dataset for scalable Bayesian inference. Typically, a Bayesian pseudocoreset is constructed by minimizing a divergence measure between the posterior conditioning on the pseudocoreset and the posterior conditioning on the full dataset. However, evaluating the divergence can be challenging, particularly for the models like deep neural networks having high-dimensional parameters. In this paper, we propose a novel Bayesian pseudocoreset construction method that operates on a function space. Unlike previous methods, which construct and match the coreset and full data posteriors in the space of model parameters (weights), our method constructs variational approximations to the coreset posterior on a function space and matches it to the full data posterior in the function space. By working directly on the function space, our method could bypass several challenges that may arise when working on a weight space, including limited scalability and multi-modality issue. Through various experiments, we demonstrate that the Bayesian pseudocoresets constructed from our method enjoys enhanced uncertainty quantification and better robustness across various model architectures.

翻译：贝叶斯伪核心集是一种紧凑的合成数据集，其总结了大规模数据集的关键信息，因此可作为贝叶斯推理的可扩展代理数据集。通常，贝叶斯伪核心集通过最小化基于伪核心集的后验与基于完整数据集的后验之间的散度度量来构建。然而，评估该散度可能存在挑战，尤其是对于深度神经网络等具有高维参数的模型。本文提出一种在函数空间上操作的新型贝叶斯伪核心集构建方法。与以往在模型参数（权重）空间构建并匹配核心集与完整数据后验的方法不同，我们的方法在函数空间构建核心集后验的变分近似，并在函数空间将其与完整数据后验进行匹配。通过直接在函数空间操作，我们的方法能够规避在权重空间操作时可能出现的多个挑战，包括有限的可扩展性和多模态问题。通过多种实验，我们证明了由该方法构建的贝叶斯伪核心集在各种模型架构上均具有增强的不确定性量化和更好的鲁棒性。