Document-level context for neural machine translation (NMT) is crucial to improve the translation consistency and cohesion, the translation of ambiguous inputs, as well as several other linguistic phenomena. Many works have been published on the topic of document-level NMT, but most restrict the system to only local context, typically including just the one or two preceding sentences as additional information. This might be enough to resolve some ambiguous inputs, but it is probably not sufficient to capture some document-level information like the topic or style of a conversation. When increasing the context size beyond just the local context, there are two challenges: (i) the~memory usage increases exponentially (ii) the translation performance starts to degrade. We argue that the widely-used attention mechanism is responsible for both issues. Therefore, we propose a constrained attention variant that focuses the attention on the most relevant parts of the sequence, while simultaneously reducing the memory consumption. For evaluation, we utilize targeted test sets in combination with novel evaluation techniques to analyze the translations in regards to specific discourse-related phenomena. We find that our approach is a good compromise between sentence-level NMT vs attending to the full context, especially in low resource scenarios.

翻译：文档级上下文对于神经机器翻译（NMT）至关重要，能够提升翻译的一致性、连贯性、歧义输入的翻译效果，以及其他多种语言现象的处理能力。目前已有大量关于文档级NMT的研究成果，但大多数系统仅局限于局部上下文，通常仅将前一个或两个句子作为额外信息。这种方法虽能解决部分歧义输入，但可能不足以捕捉对话主题或风格等文档级信息。当将上下文范围扩展至局部上下文之外时，面临两大挑战：（i）内存消耗呈指数级增长，（ii）翻译性能开始下降。我们认为广泛使用的注意力机制是导致这两个问题的根本原因。因此，我们提出了一种约束注意力变体，该机制将注意力聚焦于序列中最相关的部分，同时降低内存消耗。在评估方面，我们利用定向测试集结合新型评估技术，针对特定语篇现象对翻译结果进行分析。实验表明，我们的方法在句子级NMT与全上下文注意力机制之间取得了良好平衡，尤其在低资源场景中表现突出。