Natural behavior consists of dynamics that are complex and unpredictable, especially when trying to predict many steps into the future. While some success has been found in building representations of behavior under constrained or simplified task-based conditions, many of these models cannot be applied to free and naturalistic settings where behavior becomes increasingly hard to model. In this work, we develop a multi-task representation learning model for behavior that combines two novel components: (i) An action prediction objective that aims to predict the distribution of actions over future timesteps, and (ii) A multi-scale architecture that builds separate latent spaces to accommodate short- and long-term dynamics. After demonstrating the ability of the method to build representations of both local and global dynamics in realistic robots in varying environments and terrains, we apply our method to the MABe 2022 Multi-agent behavior challenge, where our model ranks 1st overall and on all global tasks, and 1st or 2nd on 7 out of 9 frame-level tasks. In all of these cases, we show that our model can build representations that capture the many different factors that drive behavior and solve a wide range of downstream tasks.

翻译：自然行为由复杂且不可预测的动力学构成，尤其在尝试预测未来多个时间步时尤为困难。尽管在受约束或简化任务条件下构建行为表征已取得一定成功，但许多模型无法应用于自由且自然的环境——此类场景中行为建模的难度显著增加。本研究提出一种面向行为的多任务表征学习模型，该模型融合两大创新组件：（i）动作预测目标，旨在预测未来时间步上的动作分布；（ii）多尺度架构，通过构建独立的潜在空间分别处理短期与长期动力学。在验证该方法能构建真实机器人在不同环境与地形中局部与全局动力学表征的能力后，我们将该方法应用于MABe 2022多智能体行为挑战赛。我们的模型在整体排名及所有全局任务中均位列第一，并在9个帧级任务中的7个任务上排名第一或第二。在所有案例中，该模型构建的表征均能捕捉驱动行为的多元因素，并解决广泛的后续任务。