The concept of updating a probability distribution in the light of new evidence lies at the heart of statistics and machine learning. Pearl's and Jeffrey's rule are two natural update mechanisms which lead to different outcomes, yet the similarities and differences remain mysterious. This paper clarifies their relationship in several ways: via separate descriptions of the two update mechanisms in terms of probabilistic programs and sampling semantics, and via different notions of likelihood (for Pearl and for Jeffrey). Moreover, it is shown that Jeffrey's update rule arises via variational inference. In terms of categorical probability theory, this amounts to an analysis of the situation in terms of the behaviour of the multiset functor, extended to the Kleisli category of the distribution monad.
翻译:在新证据下更新概率分布的概念是统计学和机器学习领域的核心。珍珠更新规则和杰弗里更新规则是两种产生不同结果的自然更新机制,然而两者之间的异同仍令人困惑。本文通过多种方式阐明了两者关系:通过概率程序与采样语义分别描述两种更新机制,以及基于不同似然概念(针对珍珠和杰弗里)。此外,研究表明杰弗里更新规则可通过变分推断推导得出。在范畴概率论中,这相当于从多重集函子(扩展至分布单子的克莱斯利范畴)的行为角度分析该情境。