The Frobenius norm is a frequent choice of norm for matrices. In particular, the underlying Frobenius inner product is typically used to evaluate the gradient of an objective with respect to matrix variable, such as those occuring in the training of neural networks. We provide a broader view on the Frobenius norm and inner product for linear maps or matrices, and establish their dependence on inner products in the domain and co-domain spaces. This shows that the classical Frobenius norm is merely one special element of a family of more general Frobenius-type norms. The significant extra freedom furnished by this realization can be used, among other things, to precondition neural network training.
翻译:Frobenius范数是矩阵范数中的常用选择。特别地,其对应的Frobenius内积通常用于评估关于矩阵变量的目标函数梯度(如神经网络训练中出现的梯度)。本文针对线性映射或矩阵,提出了Frobenius范数与内积的更广义视角,并建立了它们对定义域和余定义域中内积的依赖关系。这表明经典Frobenius范数仅是一类更广泛的Frobenius型范数族中的特例。这一认识所带来的显著额外自由度可用于(除其他用途外)神经网络训练的预处理。