The concept of updating a probability distribution in the light of new evidence lies at the heart of statistics and machine learning. Pearl's and Jeffrey's rule are two natural update mechanisms which lead to different outcomes, yet the similarities and differences remain mysterious. This paper clarifies their relationship in several ways: via separate descriptions of the two update mechanisms in terms of probabilistic programs and sampling semantics, and via different notions of likelihood (for Pearl and for Jeffrey). Moreover, it is shown that Jeffrey's update rule arises via variational inference. In terms of categorical probability theory, this amounts to an analysis of the situation in terms of the behaviour of the multiset functor, extended to the Kleisli category of the distribution monad.
翻译:在统计学与机器学习中,根据新证据更新概率分布的概念处于核心地位。珍珠规则与杰弗里规则是两种自然的更新机制,会产生不同结果,但二者间的异同仍显神秘。本文通过以下方式阐明其关系:基于概率程序与采样语义分别描述两种更新机制,并引入(珍珠与杰弗里意义上的)不同似然概念。进一步研究表明,杰弗里更新规则可通过变分推断导出。从范畴概率论视角看,这相当于通过多重集函子(扩展至分布单子上的克莱斯利范畴)的行为对情境进行解析。