Recently, Mutual Information (MI) has attracted attention in bounding the generalization error of Deep Neural Networks (DNNs). However, it is intractable to accurately estimate the MI in DNNs, thus most previous works have to relax the MI bound, which in turn weakens the information theoretic explanation for generalization. To address the limitation, this paper introduces a probabilistic representation of DNNs for accurately estimating the MI. Leveraging the proposed MI estimator, we validate the information theoretic explanation for generalization, and derive a tighter generalization bound than the state-of-the-art relaxations.

翻译：最近,互通信息(MI)在约束深神经网络(DNNs)的笼统错误时引起了人们的注意。 然而,准确估计DNNs的误差是难以做到的,因此,大多数以前的工作都必须放松MI约束,这反过来削弱了对概括化的信息理论解释。 为了解决这一限制,本文件引入了DNes的概率代表,以准确估计MI。 利用拟议的MI测算器,我们验证了信息理论解释的笼统化,并得出比最新放松措施更严格的概括化。