Thanks to advances in deep learning techniques, Human Pose Estimation (HPE) has achieved significant progress in natural scenarios. However, these models perform poorly in artificial scenarios such as painting and sculpture due to the domain gap, constraining the development of virtual reality and augmented reality. With the growth of model size, retraining the whole model on both natural and artificial data is computationally expensive and inefficient. Our research aims to bridge the domain gap between natural and artificial scenarios with efficient tuning strategies. Leveraging the potential of language models, we enhance the adaptability of traditional pose estimation models across diverse scenarios with a novel framework called VLPose. VLPose leverages the synergy between language and vision to extend the generalization and robustness of pose estimation models beyond the traditional domains. Our approach has demonstrated improvements of 2.26% and 3.74% on HumanArt and MSCOCO, respectively, compared to state-of-the-art tuning strategies.

翻译：得益于深度学习技术的进步，人体姿态估计（HPE）已在自然场景中取得了显著进展。然而，由于领域鸿沟的存在，这类模型在绘画和雕塑等人工场景中表现欠佳，制约了虚拟现实与增强现实技术的发展。随着模型规模的增长，在自然与人工数据上对整个模型进行重新训练在计算上成本高昂且效率低下。本研究旨在通过高效的微调策略弥合自然场景与人工场景之间的领域鸿沟。我们利用语言模型的潜力，提出名为VLPose的新型框架，增强传统姿态估计模型在不同场景下的适应性。VLPose通过语言与视觉的协同作用，将姿态估计模型的泛化能力和鲁棒性扩展至传统领域之外。与最先进的微调策略相比，我们的方法在HumanArt和MSCOCO数据集上分别提升了2.26%和3.74%。