This paper builds bridges between two families of probabilistic algorithms: (hierarchical) variational inference (VI), which is typically used to model distributions over continuous spaces, and generative flow networks (GFlowNets), which have been used for distributions over discrete structures such as graphs. We demonstrate that, in certain cases, VI algorithms are equivalent to special cases of GFlowNets in the sense of equality of expected gradients of their learning objectives. We then point out the differences between the two families and show how these differences emerge experimentally. Notably, GFlowNets, which borrow ideas from reinforcement learning, are more amenable than VI to off-policy training without the cost of high gradient variance induced by importance sampling. We argue that this property of GFlowNets can provide advantages for capturing diversity in multimodal target distributions.

翻译：本文在多类概率算法之间架起桥梁：（分层）变分推断（VI）主要应用于连续空间上的分布建模，而生成流网络（GFlowNets）则常用于图等离散结构上的分布建模。我们证明，在特定情形下，VI算法在期望学习目标梯度相等意义上等价于GFlowNets的特例。随后指出两类算法间的差异，并通过实验揭示这些差异的具体表现形式。值得注意的是，借鉴强化学习思想的GFlowNets比VI更易于实现离策略训练，且无需承担重要性采样导致的高梯度方差代价。我们认为，GFlowNets的这一特性有助于捕获多模态目标分布中的多样性。