The Fisher information matrix (FIM) has been applied to the realm of deep learning. It is closely related to the loss landscape, the variance of the parameters, second order optimization, and deep learning theory. However, the exact FIM is either unavailable in closed form or too expensive to compute. In practice, it is almost always estimated based on empirical samples. We investigate two such estimators based on two equivalent representations of the FIM -- both unbiased and consistent with respect to the underlying "true" FIM. Their estimation quality is characterized by their variance given in closed form. We bound their variances and analyze how the parametric structure of a deep neural network can impact the variance. We discuss the meaning of this variance measure and our bounds in the context of deep learning.

翻译：渔业信息矩阵(FIM)已应用于深层学习领域,与损失情况、参数差异、第二顺序优化和深层学习理论密切相关,然而,确切的FIM要么没有封闭形式,要么过于昂贵,无法计算。实际上,几乎总是根据经验样本估算。我们根据FIM的两个等同表述调查了两个这样的估算数据 -- -- 既不带偏见,也符合基本“真实”FIM。其估算质量的特点是以封闭形式给出的差异。我们限制其差异,分析深神经网络的参数结构如何影响差异。我们从深层学习的角度讨论这一差异尺度的含义和界限。