Hand pose estimation (HPE) is a task that predicts and describes the hand poses from images or video frames. When HPE models estimate hand poses captured in a laboratory or under controlled environments, they normally deliver good performance. However, the real-world environment is complex, and various uncertainties may happen, which could degrade the performance of HPE models. For example, the hands could be occluded, the visibility of hands could be reduced by imperfect exposure rate, and the contour of hands prone to be blurred during fast hand movements. In this work, we adopt metamorphic testing to evaluate the robustness of HPE models and provide suggestions on the choice of HPE models for different applications. The robustness evaluation was conducted on four state-of-the-art models, namely MediaPipe hands, OpenPose, BodyHands, and NSRM hand. We found that on average more than 80\% of the hands could not be identified by BodyHands, and at least 50\% of hands could not be identified by MediaPipe hands when diagonal motion blur is introduced, while an average of more than 50\% of strongly underexposed hands could not be correctly estimated by NSRM hand. Similarly, applying occlusions on only four hand joints will also largely degrade the performance of these models. The experimental results show that occlusions, illumination variations, and motion blur are the main obstacles to the performance of existing HPE models. These findings may pave the way for researchers to improve the performance and robustness of hand pose estimation models and their applications.

翻译：手部姿态估计（HPE）是一项从图像或视频帧中预测并描述手部姿态的任务。当HPE模型在实验室或受控环境下估计手部姿态时，通常能表现出良好的性能。然而，现实环境复杂多变，各种不确定性因素可能导致HPE模型性能下降。例如，手部可能被遮挡，曝光率不足会降低手部可见度，快速手部运动时手部轮廓易产生模糊。本研究采用蜕变测试评估HPE模型的鲁棒性，并为不同应用场景下HPE模型的选择提供建议。我们对四种最先进模型（MediaPipe Hands、OpenPose、BodyHands和NSRM Hand）进行了鲁棒性评估。研究发现，当引入对角线运动模糊时，BodyHands平均有超过80%的手部无法识别，MediaPipe Hands至少50%的手部无法识别；而NSRM Hand对强曝光不足的手部平均有超过50%无法正确估计。类似地，仅对四个手部关节施加遮挡也会大幅降低这些模型的性能。实验结果表明，遮挡、光照变化和运动模糊是现有HPE模型性能的主要障碍。这些发现可为研究人员改进手部姿态估计模型及其应用的性能和鲁棒性铺平道路。