Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes. We propose Recurrent Independent Mechanisms (RIMs), a new recurrent architecture in which multiple groups of recurrent cells operate with nearly independent transition dynamics, communicate only sparingly through the bottleneck of attention, and are only updated at time steps where they are most relevant. We show that this leads to specialization amongst the RIMs, which in turn allows for dramatically improved generalization on tasks where some factors of variation differ systematically between training and evaluation.

翻译：反映环境动态的学习模块结构可导致更好地概括和稳健地改变,这些变化只影响到少数根本原因。 我们提议设立经常性独立机制(RIMs),这是一个新的经常性结构,在这个结构中,多组经常细胞以几乎独立的过渡动态运作,只是通过受关注的瓶颈进行零星交流,并且只是在最相关的时间步骤上更新。我们表明,这导致RIMs之间的专业化,这反过来又使得在培训和评估之间有系统差异的一些因素时,可以大大改进任务的一般化。