Modern Neural Machine Translation systems exhibit strong performance in several different languages and are constantly improving. Their ability to learn continuously is, however, still severely limited by the catastrophic forgetting issue. In this work, we leverage a key property of encoder-decoder Transformers, i.e. their generative ability, to propose a novel approach to continually learning Neural Machine Translation systems. We show how this can effectively learn on a stream of experiences comprising different languages, by leveraging a replay memory populated by using the model itself as a generator of parallel sentences. We empirically demonstrate that our approach can counteract catastrophic forgetting without requiring explicit memorization of training data. Code will be publicly available upon publication. Code: https://github.com/m-resta/sg-rep

翻译：现代神经机器翻译系统在多种语言中展现出强劲性能，且持续改进中。然而，其持续学习能力仍受制于灾难性遗忘问题。本研究利用编码器-解码器Transformer的关键特性——即其生成能力，提出一种持续学习神经机器翻译系统的新方法。我们展示了该方法如何通过利用模型自身生成的平行句子填充回放记忆，在包含不同语言的经验流中实现高效学习。实验证明，该方法无需显式记忆训练数据即可有效对抗灾难性遗忘。代码将于论文发表后公开。代码地址：https://github.com/m-resta/sg-rep