Motion forecasting has become an increasingly critical component of autonomous robotic systems. Onboard compute budgets typically limit the accuracy of real-time systems. In this work we propose methods of improving motion forecasting systems subject to limited compute budgets by combining model ensemble and distillation techniques. The use of ensembles of deep neural networks has been shown to improve generalization accuracy in many application domains. We first demonstrate significant performance gains by creating a large ensemble of optimized single models. We then develop a generalized framework to distill motion forecasting model ensembles into small student models which retain high performance with a fraction of the computing cost. For this study we focus on the task of motion forecasting using real world data from autonomous driving systems. We develop ensemble models that are very competitive on the Waymo Open Motion Dataset (WOMD) and Argoverse leaderboards. From these ensembles, we train distilled student models which have high performance at a fraction of the compute costs. These experiments demonstrate distillation from ensembles as an effective method for improving accuracy of predictive models for robotic systems with limited compute budgets.

翻译：运动预测已成为自主机器人系统中日益关键的组成部分。机载计算预算通常限制了实时系统的精度。本研究提出在计算预算受限的条件下，通过结合模型集成与蒸馏技术来改进运动预测系统的方法。深度神经网络集成的应用已在多个领域被证明能够提升泛化精度。我们首先通过构建大规模优化单模型集成，展示了显著的性能提升。随后开发了一个通用框架，将运动预测模型集成蒸馏为小型学生模型，这些模型在保持高性能的同时仅需极低计算成本。本研究聚焦于基于自动驾驶系统真实数据的运动预测任务。我们开发的集成模型在Waymo开放运动数据集（WOMD）和Argoverse排行榜上展现出强劲竞争力。从这些集成中，我们训练出计算成本极低且性能卓越的蒸馏学生模型。这些实验证明，对于计算预算受限的机器人系统，集成蒸馏是提升预测模型精度的有效方法。