Many complex tasks can be decomposed into simpler, independent parts. Discovering such underlying compositional structure has the potential to enable compositional generalization. Despite progress, our most powerful systems struggle to compose flexibly. It therefore seems natural to make models more modular to help capture the compositional nature of many tasks. However, it is unclear under which circumstances modular systems can discover hidden compositional structure. To shed light on this question, we study a teacher-student setting with a modular teacher where we have full control over the composition of ground truth modules. This allows us to relate the problem of compositional generalization to that of identification of the underlying modules. In particular we study modularity in hypernetworks representing a general class of multiplicative interactions. We show theoretically that identification up to linear transformation purely from demonstrations is possible without having to learn an exponential number of module combinations. We further demonstrate empirically that under the theoretically identified conditions, meta-learning from finite data can discover modular policies that generalize compositionally in a number of complex environments.

翻译：许多复杂任务可被分解为更简单、相互独立的子部分。发现这种潜在的组合结构有望实现组合泛化。尽管取得了进展，但现有最强系统在灵活组合方面仍存在困难。因此，使模型更具模块化以帮助捕捉许多任务的组合本质似乎顺理成章。然而，尚不明确在何种条件下模块化系统能够发现隐藏的组合结构。为阐明该问题，我们研究了包含模块化教师模型的师生设定，其中可完全控制真实模块的组合方式。这使得我们能够将组合泛化问题与潜在模块的识别问题联系起来。具体而言，我们研究了代表广义乘法交互的超网络中的模块化特性。理论证明，仅通过示范进行线性变换下的模块识别是可行的，无需学习指数级数量的模块组合。进一步通过实证表明，在理论识别的条件下，基于有限数据的元学习能够发现可在多种复杂环境中实现组合泛化的模块化策略。