The task of human pose estimation (HPE) deals with the ill-posed problem of estimating the 3D position of human joints directly from images and videos. In recent literature, most of the works tackle the problem mostly by using convolutional neural networks (CNNs), which are capable of achieving state-of-the-art results in most datasets. We show how most neural networks are not able to generalize well when the camera is subject to significant viewpoint changes. This behaviour emerges because CNNs lack the capability of modelling viewpoint equivariance, while they rather rely on viewpoint invariance, resulting in high data dependency. Recently, capsule networks (CapsNets) have been proposed in the multi-class classification field as a solution to the viewpoint equivariance issue, reducing both the size and complexity of both the training datasets and the network itself. In this work, we show how capsule networks can be adopted to achieve viewpoint equivariance in human pose estimation. We propose a novel end-to-end viewpoint-equivariant capsule autoencoder that employs a fast Variational Bayes routing and matrix capsules. We achieve state-of-the-art results for multiple tasks and datasets while retaining other desirable properties, such as greater generalization capabilities when changing viewpoints, lower data dependency and fast inference. Additionally, by modelling each joint as a capsule, the hierarchical and geometrical structure of the overall pose is retained in the feature space, independently from the viewpoint. We further test our network on multiple datasets, both in the RGB and depth domain, from seen and unseen viewpoints and in the viewpoint transfer task.

翻译：人体姿态估计（HPE）任务旨在解决从图像和视频中直接估计人体关节三维位置的病态问题。近年来，多数研究主要采用卷积神经网络（CNN）处理该问题，此类网络能在大多数数据集上取得顶尖结果。然而我们发现，当相机视角发生显著变化时，多数神经网络难以实现良好泛化。这一现象源于CNN缺乏建模视角等变性（viewpoint equivariance）的能力，而更依赖视角不变性（viewpoint invariance），导致对数据高度依赖。近期，胶囊网络（CapsNet）在多类分类领域被提出以解决视角等变性问题，可同时降低训练数据集和网络本身的大小与复杂度。本研究展示如何采用胶囊网络实现人体姿态估计中的视角等变性。我们提出一种新型端到端视角等变胶囊自编码器，该模型采用快速变分贝叶斯路由（fast Variational Bayes routing）与矩阵胶囊。在多项任务和数据集上，我们既取得顶尖结果，同时保持其他优秀特性：视角改变时更强的泛化能力、更低的数据依赖性以及快速推理。此外，通过将每个关节建模为胶囊，整体姿态的层级几何结构可在特征空间中保持与视角无关。我们进一步在RGB与深度域的多类数据集上，对可见/不可见视角及视角迁移任务进行测试。