Recurrent neural networks have flourished in many areas. Consequently, we can see new RNN cells being developed continuously, usually by creating or using gates in a new, original way. But what if we told you that gates in RNNs are redundant? In this paper, we propose a new recurrent cell called Residual Recurrent Unit (RRU) which beats traditional cells and does not employ a single gate. It is based on the residual shortcut connection, linear transformations, ReLU, and normalization. To evaluate our cell's effectiveness, we compare its performance against the widely-used GRU and LSTM cells and the recently proposed Mogrifier LSTM on several tasks including, polyphonic music modeling, language modeling, and sentiment analysis. Our experiments show that RRU outperforms the traditional gated units on most of these tasks. Also, it has better robustness to parameter selection, allowing immediate application in new tasks without much tuning. We have implemented the RRU in TensorFlow, and the code is made available at https://github.com/LUMII-Syslab/RRU .

翻译：循环神经网络已在众多领域取得显著成果。因此，我们不断看到新型RNN单元被开发出来，通常是通过以新颖独创的方式创建或使用门控机制。但如果我们告诉您RNN中的门控机制是多余的，您会作何感想？本文提出一种名为残差循环单元（RRU）的新型循环单元，该单元无需任何门控即可超越传统单元结构。该单元基于残差捷径连接、线性变换、ReLU激活函数和归一化技术构建。为评估该单元的有效性，我们将其与广泛使用的GRU和LSTM单元以及近期提出的Mogrifier LSTM进行多任务性能对比，涵盖复调音乐建模、语言建模和情感分析等任务。实验表明，RRU在大多数任务中均优于传统门控单元。此外，该单元对参数选择具有更好的鲁棒性，可直接应用于新任务而无需大量调参。我们已在TensorFlow中实现RRU，代码开源于https://github.com/LUMII-Syslab/RRU。