The task of collaborative human pose forecasting stands for predicting the future poses of multiple interacting people, given those in previous frames. Predicting two people in interaction, instead of each separately, promises better performance, due to their body-body motion correlations. But the task has remained so far primarily unexplored. In this paper, we review the progress in human pose forecasting and provide an in-depth assessment of the single-person practices that perform best for 2-body collaborative motion forecasting. Our study confirms the positive impact of frequency input representations, space-time separable and fully-learnable interaction adjacencies for the encoding GCN and FC decoding. Other single-person practices do not transfer to 2-body, so the proposed best ones do not include hierarchical body modeling or attention-based interaction encoding. We further contribute a novel initialization procedure for the 2-body spatial interaction parameters of the encoder, which benefits performance and stability. Altogether, our proposed 2-body pose forecasting best practices yield a performance improvement of 21.9% over the state-of-the-art on the most recent ExPI dataset, whereby the novel initialization accounts for 3.5%. See our project page at https://www.pinlab.org/bestpractices2body
翻译:协作式人体姿态预测任务旨在根据前几帧中多个交互个体的姿态,预测其未来帧中的姿态。与分别预测每个人的姿态相比,预测两个交互个体的姿态因人体间的运动关联性而有望获得更优性能,但该任务至今尚未得到充分探索。本文综述了人体姿态预测的研究进展,并深度评估了在双人协作运动预测中表现最佳的单人方法。研究证实了频率输入表示、时空可分离的全可学习交互邻接矩阵在编码图卷积网络与全连接解码中的积极影响。其他单人方法无法迁移至双人场景,因此所提出的最佳实践不包括层次化人体建模或基于注意力的交互编码。我们进一步提出了一种新颖的初始化方法用于编码器中双人空间交互参数的设置,该方法提升了性能与稳定性。综合而言,所提出的双人姿态预测最佳实践在最新ExPI数据集上相较于现有最优方法实现了21.9%的性能提升,其中新型初始化方法贡献了3.5%的改进。详见项目页面:https://www.pinlab.org/bestpractices2body