The well known phenomenon of exploding and vanishing gradients in deep neural networks is analyzed using multiplicative ergodic theory. The effect of adding a residual connection is explained in this context. Specifically, a characterization of Liapunov exponents due to Furstenberg and Kifer is exploited in order to make a precise statement about the Liapunov spectrum and the effect of residual connections on it.

翻译：利用乘性遍历理论分析了深度神经网络中众所周知的梯度爆炸与消失现象，并在此框架下解释了添加残差连接的影响。具体而言，借助Furstenberg和Kifer提出的李雅普诺夫指数刻画方法，本文精确阐述了李雅普诺夫谱及其受残差连接影响的效应。