We propose autophagy penalized likelihood estimation (PLE), an unbiased alternative to maximum likelihood estimation (MLE) which is more fair and less susceptible to model autophagy disorder (madness). Model autophagy refers to models trained on their own output; PLE ensures the statistics of these outputs coincide with the data statistics. This enables PLE to be statistically unbiased in certain scenarios where MLE is biased. When biased, MLE unfairly penalizes minority classes in unbalanced datasets and exacerbates the recently discovered issue of self-consuming generative modeling. Theoretical and empirical results show that 1) PLE is more fair to minority classes and 2) PLE is more stable in a self-consumed setting. Furthermore, we provide a scalable and portable implementation of PLE with a hypernetwork framework, allowing existing deep learning architectures to be easily trained with PLE. Finally, we show PLE can bridge the gap between Bayesian and frequentist paradigms in statistics.

翻译：我们提出自噬惩罚似然估计（PLE），作为最大似然估计（MLE）的无偏替代方案，该方法更具公平性且对模型自噬紊乱（madness）具有更强的鲁棒性。模型自噬指模型基于自身输出进行训练的现象；PLE确保这些输出的统计特性与数据统计特性保持一致。这使得PLE在某些MLE存在偏差的场景中能够保持统计无偏性。当存在偏差时，MLE会不公平地惩罚不平衡数据集中的少数类别，并加剧近期发现的自消耗生成建模问题。理论与实证结果表明：1）PLE对少数类别更具公平性；2）在自消耗训练环境中PLE具有更优的稳定性。此外，我们通过超网络框架提供了可扩展、可移植的PLE实现方案，使得现有深度学习架构能够便捷地采用PLE进行训练。最后，我们证明PLE能够弥合统计学中贝叶斯学派与频率学派之间的理论鸿沟。