Bayes' theorem admits an information processing interpretation due to Zellner (1988): under the Shannon information criterion, the posterior is the unique rule that processes prior and data information without information loss. We revisit these ideas from the perspective of information deletion. Given a posterior based on a complete dataset, what distribution should replace it when a subset of the data is removed? For ordinary Bayesian inference, we show that the corresponding information deletion objective is uniquely minimized by the leave-data-out posterior. We then extend the framework beyond likelihood-based inference from Bayes to the generalized Bayesian updating of Bissiri et al.(2016} based on loss functions. Requiring deletion to be coherent under sequential removal of loss contributions yields a deletion rule that exactly recovers the generalized posterior based only on the retained information. We further develop an axiomatic theory, beyond probabilistic inference, for possibilistic deletion and show that the same information-deletion perspective leads to a principled rule for exact possibilistic unlearning. Finally, restricting these exact deletion principles to tractable families leads naturally to variational Bayesian, generalized Bayesian, and maxitive possibilistic unlearning.

翻译：暂无翻译