A Deep Neural Network (DNN) is a composite function of vector-valued functions, and in order to train a DNN, it is necessary to calculate the gradient of the loss function with respect to all parameters. This calculation can be a non-trivial task because the loss function of a DNN is a composition of several nonlinear functions, each with numerous parameters. The Backpropagation (BP) algorithm leverages the composite structure of the DNN to efficiently compute the gradient. As a result, the number of layers in the network does not significantly impact the complexity of the calculation. The objective of this paper is to express the gradient of the loss function in terms of a matrix multiplication using the Jacobian operator. This can be achieved by considering the total derivative of each layer with respect to its parameters and expressing it as a Jacobian matrix. The gradient can then be represented as the matrix product of these Jacobian matrices. This approach is valid because the chain rule can be applied to a composition of vector-valued functions, and the use of Jacobian matrices allows for the incorporation of multiple inputs and outputs. By providing concise mathematical justifications, the results can be made understandable and useful to a broad audience from various disciplines.

翻译：深度神经网络(DNN)是向量值函数的复合函数，为训练DNN需计算损失函数对所有参数的梯度。由于DNN的损失函数由多个非线性函数复合而成，且每个函数包含大量参数，该计算任务颇具挑战性。反向传播(BP)算法利用DNN的复合结构高效计算梯度，使得网络层数对计算复杂度的影响不显著。本文旨在通过雅可比算子将损失函数梯度表示为矩阵乘法形式。具体而言，通过考虑各层关于其参数的全导数并将其表达为雅可比矩阵，梯度即可表示为这些雅可比矩阵的矩阵乘积。此方法的可行性源于链式法则可应用于向量值函数的复合结构，而雅可比矩阵的使用则能处理多输入多输出情形。通过提供简洁的数学推导，本文结果可被不同学科背景的读者理解并应用。