The light gated recurrent units (Li-GRU) is well-known for achieving impressive results in automatic speech recognition (ASR) tasks while being lighter and faster to train than a standard gated recurrent units (GRU). However, the unbounded nature of its rectified linear unit on the candidate recurrent gate induces an important gradient exploding phenomenon disrupting the training process and preventing it from being applied to famous datasets. In this paper, we theoretically and empirically derive the necessary conditions for its stability as well as engineering mechanisms to speed up by a factor of five its training time, hence introducing a novel version of this architecture named SLi-GRU. Then, we evaluate its performance both on a toy task illustrating its newly acquired capabilities and a set of three different ASR datasets demonstrating lower word error rates compared to more complex recurrent neural networks.

翻译：轻量门控循环单元（Li-GRU）以其在自动语音识别（ASR）任务中取得显著成果且比标准门控循环单元（GRU）更轻量、训练更快速而闻名。然而，其候选循环门上的线性整流单元无界特性会导致严重的梯度爆炸现象，这不仅干扰训练过程，还阻碍其在知名数据集上的应用。本文从理论与实证角度推导了其稳定性的必要条件，并提出了将训练时间提升五倍的工程机制，由此引入了该架构的新版本——SLi-GRU。随后，我们通过一项展示其新增能力的玩具任务以及三个不同ASR数据集对其性能进行评估，结果表明与更复杂的循环神经网络相比，该方法实现了更低的词错误率。