Human beings use compositionality to generalise from past experiences to novel experiences. We assume a separation of our experiences into fundamental atomic components that can be recombined in novel ways to support our ability to engage with novel experiences. We frame this as the ability to learn to generalise compositionally, and we will refer to behaviours making use of this ability as compositional learning behaviours (CLBs). A central problem to learning CLBs is the resolution of a binding problem (BP). While it is another feat of intelligence that human beings perform with ease, it is not the case for state-of-the-art artificial agents. Thus, in order to build artificial agents able to collaborate with human beings, we propose to develop a novel benchmark to investigate agents' abilities to exhibit CLBs by solving a domain-agnostic version of the BP. We take inspiration from the language emergence and grounding framework of referential games and propose a meta-learning extension of referential games, entitled Meta-Referential Games, and use this framework to build our benchmark, the Symbolic Behaviour Benchmark (S2B). We provide baseline results and error analysis showing that our benchmark is a compelling challenge that we hope will spur the research community towards developing more capable artificial agents.

翻译：人类利用组合性从过去的经验泛化到新经验。我们假设经验可以分解为基本的原子组件，这些组件能以新颖方式重新组合，从而支持我们应对新经验的能力。我们将此能力定义为学习组合泛化，并将利用该能力的行为称为组合学习行为。学习组合学习行为的一个核心问题是解决绑定问题。尽管人类能够轻松实现这一智能壮举，但最先进的人工智能代理却难以企及。因此，为了构建能与人类协作的人工智能代理，我们提出开发一种新型基准测试，通过解决领域无关版本的绑定问题来评估代理展现组合学习行为的能力。我们从参考游戏的语言涌现与基础框架中获得灵感，提出参考游戏的元学习扩展——元参考游戏，并利用该框架构建基准测试——符号行为基准。我们提供的基线结果与误差分析表明，该基准测试具有重大挑战性，有望推动研究社区开发出更强大的人工智能代理。