Control variates are variance reduction techniques for Monte Carlo estimators. They play a critical role in improving Monte Carlo estimators in scientific and machine learning applications that involve computationally expensive integrals. We introduce multilevel control functionals (MLCFs), a novel and widely applicable extension of control variates that combines non-parametric Stein-based control variates with multi-fidelity methods. We show that when the integrand and the density are smooth, and when the dimensionality is not very high, MLCFs enjoy a faster convergence rate. We provide both theoretical analysis and empirical assessments on differential equation examples, including Bayesian inference for ecological models, to demonstrate the effectiveness of our proposed approach. Furthermore, we extend MLCFs for variational inference, and demonstrate improved performance empirically through Bayesian neural network examples.

翻译：控制变量是用于蒙特卡洛估计器的方差缩减技术。在涉及计算成本高昂的积分的科学和机器学习应用中，它们对于改进蒙特卡洛估计器起着至关重要的作用。我们提出了多级控制泛函，这是一种新颖且广泛适用的控制变量扩展方法，它将基于Stein方法的非参数控制变量与多保真度方法相结合。我们证明，当被积函数和概率密度函数光滑，且维数不太高时，多级控制泛函具有更快的收敛速率。我们通过微分方程示例（包括生态模型的贝叶斯推断）提供了理论分析和实证评估，以证明我们所提方法的有效性。此外，我们将多级控制泛函扩展至变分推断，并通过贝叶斯神经网络示例实证展示了其性能提升。