The current 3D human pose estimators face challenges in adapting to new datasets due to the scarcity of 2D-3D pose pairs in target domain training sets. We present the \textit{Multi-Hypothesis \textbf{P}ose \textbf{Syn}thesis \textbf{D}omain \textbf{A}daptation} (\textbf{PoSynDA}) framework to overcome this issue without extensive target domain annotation. Utilizing a diffusion-centric structure, PoSynDA simulates the 3D pose distribution in the target domain, filling the data diversity gap. By incorporating a multi-hypothesis network, it creates diverse pose hypotheses and aligns them with the target domain. Target-specific source augmentation obtains the target domain distribution data from the source domain by decoupling the scale and position parameters. The teacher-student paradigm and low-rank adaptation further refine the process. PoSynDA demonstrates competitive performance on benchmarks, such as Human3.6M, MPI-INF-3DHP, and 3DPW, even comparable with the target-trained MixSTE model~\cite{zhang2022mixste}. This work paves the way for the practical application of 3D human pose estimation. The code is available at https://github.com/hbing-l/PoSynDA.
翻译:当前三维人体姿态估计器面临适应新数据集的挑战,原因在于目标域训练集中缺乏二维-三维姿态配对数据。我们提出多假设姿态合成域自适应(PoSynDA)框架,在不依赖大量目标域标注的情况下解决该问题。基于扩散模型核心结构,PoSynDA模拟目标域的三维姿态分布,填补数据多样性缺口。通过集成多假设网络,该框架生成多样化姿态假设并将其与目标域对齐。面向目标的源域增强通过解耦尺度与位置参数,从源域获取目标域分布数据。教师-学生范式与低秩自适应进一步优化该过程。PoSynDA在Human3.6M、MPI-INF-3DHP、3DPW等基准数据集上展现出竞争性性能,甚至可与目标域训练的MixSTE模型相媲美。该工作为三维人体姿态估计的实际应用铺平道路。代码开源于https://github.com/hbing-l/PoSynDA。