Transformer encoders contextualize token representations by attending to all other tokens at each layer, leading to quadratic increase in compute effort with the input length. In practice, however, the input text of many NLP tasks can be seen as a sequence of related segments (e.g., the sequence of sentences within a passage, or the hypothesis and premise in NLI). While attending across these segments is highly beneficial for many tasks, we hypothesize that this interaction can be delayed until later encoding stages. To this end, we introduce Layer-Adjustable Interactions in Transformers (LAIT). Within LAIT, segmented inputs are first encoded independently, and then jointly. This partial two-tower architecture bridges the gap between a Dual Encoder's ability to pre-compute representations for segments and a fully self-attentive Transformer's capacity to model cross-segment attention. The LAIT framework effectively leverages existing pretrained Transformers and converts them into the hybrid of the two aforementioned architectures, allowing for easy and intuitive control over the performance-efficiency tradeoff. Experimenting on a wide range of NLP tasks, we find LAIT able to reduce 30-50% of the attention FLOPs on many tasks, while preserving high accuracy; in some practical settings, LAIT could reduce actual latency by orders of magnitude.

翻译：论文摘要：Transformer编码器通过在每个层级对所有其他token进行注意力计算来实现token表示的上下文化，导致计算量随输入长度呈二次增长。然而在实际应用中，众多自然语言处理任务的输入文本可视为由多个相关片段组成的序列（例如段落中的句子序列，或自然语言推理中的前提与假设）。尽管跨片段注意力对许多任务极为有益，但我们假设这种交互可以被推迟到后期编码阶段。为此，我们提出层级可调节交互Transformer（LAIT）。在LAIT框架中，分段输入首先被独立编码，随后进行联合编码。这种部分双塔架构填补了双编码器预计算片段表示能力与全自注意力Transformer建模跨片段注意力能力之间的鸿沟。LAIT框架能有效利用预训练Transformer模型，将其转化为前述两种架构的混合体，从而实现对性能-效率权衡的直观便捷控制。通过在多种自然语言处理任务上的实验，我们发现LAIT能在保持高准确率的前提下，减少30-50%的注意力浮点运算量；在某些实际场景中，LAIT可将实际延迟降低数个数量级。