The self-attention mechanism in the transformer architecture is capable of capturing long-range dependencies and it is the main reason behind its effectiveness in processing sequential data. Nevertheless, despite their success, transformers have two significant drawbacks that still limit their broader applicability: (1) In order to remember past information, the self-attention mechanism requires access to the whole history to be provided as context. (2) The inference cost in transformers is expensive. In this paper we introduce recurrent alternatives to the transformer self-attention mechanism that offer a context-independent inference cost, leverage long-range dependencies effectively, and perform well in practice. We evaluate our approaches in reinforcement learning problems where the aforementioned computational limitations make the application of transformers nearly infeasible. We quantify the impact of the different components of our architecture in a diagnostic environment and assess performance gains in 2D and 3D pixel-based partially-observable environments. When compared to a state-of-the-art architecture, GTrXL, inference in our approach is at least 40% cheaper while reducing memory use in more than 50%. Our approach either performs similarly or better than GTrXL, improving more than 37% upon GTrXL performance on harder tasks.

翻译：Transformer架构中的自注意力机制能够捕捉长距离依赖关系，这是其处理序列数据有效性的主要成因。然而，尽管取得了成功，Transformer仍存在两个显著缺陷限制了其广泛适用性：（1）为记忆过去信息，自注意力机制需要访问完整历史作为上下文。（2）Transformer的推理成本高昂。本文提出Transformer自注意力机制的循环替代方案，该方案具有与上下文无关的推理成本，可有效利用长距离依赖关系，并在实际应用中表现良好。我们在强化学习问题中评估所提方法——这类问题中前述计算限制使Transformer几乎无法应用。我们在诊断环境中量化架构各组件的贡献度，并在基于像素的2D和3D部分可观测环境中评估性能提升。与当前最优架构GTrXL相比，本方法的推理成本至少降低40%，同时内存使用减少超过50%。本方法在性能上与GTrXL相当或更优，在困难任务上较GTrXL提升超过37%。