Importance sampling is a popular technique in Bayesian inference: by reweighting samples drawn from a proposal distribution we are able to obtain samples and moment estimates from a Bayesian posterior over some $n$ latent variables. Recent work, however, indicates that importance sampling scales poorly -- in order to accurately approximate the true posterior, the required number of importance samples grows is exponential in the number of latent variables [Chatterjee and Diaconis, 2018]. Massively parallel importance sampling works around this issue by drawing $K$ samples for each of the $n$ latent variables and reasoning about all $K^n$ combinations of latent samples. In principle, we can reason efficiently over $K^n$ combinations of samples by exploiting conditional independencies in the generative model. However, in practice this requires complex algorithms that traverse backwards through the graphical model, and we need separate backward traversals for each computation (posterior expectations, marginals and samples). Our contribution is to exploit the source term trick from physics to entirely avoid the need to hand-write backward traversals. Instead, we demonstrate how to simply and easily compute all the required quantities -- posterior expectations, marginals and samples -- by differentiating through a slightly modified marginal likelihood estimator.

翻译：重要性采样是贝叶斯推断中一种常用技术：通过对从提议分布中抽取的样本进行重加权，我们能够获得关于$n$个潜在变量的贝叶斯后验的样本和矩估计。然而，近期研究表明重要性采样的可扩展性较差——为了准确近似真实后验，所需重要性样本数量随潜在变量数量呈指数增长[Chatterjee and Diaconis, 2018]。大规模并行重要性采样通过为每个$n$个潜在变量抽取$K$个样本并考虑所有$K^n$种潜在样本组合来规避此问题。原则上，通过利用生成模型中的条件独立性，我们可以高效处理$K^n$种样本组合。但在实践中，这需要复杂的算法沿图模型反向遍历，且每次计算（后验期望、边缘分布和样本）都需要独立的反向遍历。我们的贡献在于借鉴物理学中的源项技巧，完全避免了手动编写反向遍历的需求。取而代之，我们展示了如何通过对略微修改的边际似然估计器进行自动微分，简单便捷地计算所有所需量——后验期望、边缘分布和样本。