We present a novel method for initializing layers of tensorized neural networks in a way that avoids the explosion of the parameters of the matrix it emulates. The method is intended for layers with a high number of nodes in which there is a connection to the input or output of all or most of the nodes. The core of this method is the use of the Frobenius norm of this layer in an iterative partial form, so that it has to be finite and within a certain range. This norm is efficient to compute, fully or partially for most cases of interest. We apply the method to different layers and check its performance. We create a Python function to run it on an arbitrary layer, available in a Jupyter Notebook in the i3BQuantum repository: https://github.com/i3BQuantumTeam/Q4Real/blob/e07c827651ef16bcf74590ab965ea3985143f891/Quantum-Inspired%20Variational%20Methods/Normalization_process.ipynb
翻译:我们提出了一种新颖的方法,用于初始化张量神经网络的各层,该方法避免了其所模拟矩阵参数的爆炸式增长。该方法适用于具有大量节点且所有或大部分节点与输入或输出存在连接的层。其核心是迭代地部分使用该层的弗罗贝尼乌斯范数,以确保该范数有限且处于特定范围内。对于大多数感兴趣的情形,该范数可高效地完全或部分计算。我们将该方法应用于不同层并检验其性能。我们创建了一个Python函数,可在任意层上运行该函数,该函数位于i3BQuantum仓库的Jupyter Notebook中:https://github.com/i3BQuantumTeam/Q4Real/blob/e07c827651ef16bcf74590ab965ea3985143f891/Quantum-Inspired%20Variational%20Methods/Normalization_process.ipynb