We propose bandit importance sampling (BIS), a powerful importance sampling framework tailored for settings in which evaluating the target density is computationally expensive. BIS facilitates accurate sampling while minimizing the required number of target-density evaluations. In contrast to adaptive importance sampling, which optimizes a proposal distribution, BIS directly optimizes the set of samples through a sequential selection process driven by multi-armed bandits. BIS serves as a general framework that accommodates user-defined bandit strategies. Theoretically, the weak convergence of the weighted samples, and thus the consistency of the Monte Carlo estimator, is established regardless of the specific strategy employed. In this paper, we present a practical strategy that leverages Gaussian process surrogates to guide sample selection, adapting the principles of Bayesian optimization for sampling. Comprehensive numerical studies demonstrate the superior performance of BIS across multimodal, heavy-tailed distributions, and real-world Bayesian inference tasks involving Markov random fields.

翻译：我们提出了一种赌博机重要性采样（BIS）框架，该框架专为目标密度函数评估计算成本高昂的场景而设计。BIS 能够在最小化目标密度函数评估次数的前提下，实现精确采样。与通过优化建议分布来工作的自适应重要性采样不同，BIS 通过一个由多臂赌博机驱动的序贯选择过程，直接优化样本集合。BIS 作为一个通用框架，能够容纳用户自定义的赌博机策略。理论上，无论采用何种具体策略，加权样本的弱收敛性以及蒙特卡洛估计量的一致性均能得到保证。在本文中，我们提出了一种实用策略，该策略利用高斯过程代理模型来指导样本选择，将贝叶斯优化的原理应用于采样任务。全面的数值研究表明，BIS 在处理多峰分布、重尾分布以及涉及马尔可夫随机场的实际贝叶斯推断任务中均表现出卓越的性能。