Recently, transformer-based methods have gained significant success in sequential 2D-to-3D lifting human pose estimation. As a pioneering work, PoseFormer captures spatial relations of human joints in each video frame and human dynamics across frames with cascaded transformer layers and has achieved impressive performance. However, in real scenarios, the performance of PoseFormer and its follow-ups is limited by two factors: (a) The length of the input joint sequence; (b) The quality of 2D joint detection. Existing methods typically apply self-attention to all frames of the input sequence, causing a huge computational burden when the frame number is increased to obtain advanced estimation accuracy, and they are not robust to noise naturally brought by the limited capability of 2D joint detectors. In this paper, we propose PoseFormerV2, which exploits a compact representation of lengthy skeleton sequences in the frequency domain to efficiently scale up the receptive field and boost robustness to noisy 2D joint detection. With minimum modifications to PoseFormer, the proposed method effectively fuses features both in the time domain and frequency domain, enjoying a better speed-accuracy trade-off than its precursor. Extensive experiments on two benchmark datasets (i.e., Human3.6M and MPI-INF-3DHP) demonstrate that the proposed approach significantly outperforms the original PoseFormer and other transformer-based variants. Code is released at \url{https://github.com/QitaoZhao/PoseFormerV2}.

翻译：近年来，基于Transformer的方法在从2D到3D的时序人体姿态估计任务中取得了显著成功。作为开创性工作，PoseFormer通过级联Transformer层捕获视频帧内人体关节的空间关系及帧间动态信息，展现了卓越性能。然而在真实场景中，PoseFormer及其后续方法的表现受到两个因素制约：(a)输入关节序列的长度；(b)2D关节检测的质量。现有方法通常对所有输入帧应用自注意力机制，当为提升估计精度而增加帧数时，会带来巨大的计算负担，且对因2D关节检测器能力有限而自然引入的噪声缺乏鲁棒性。本文提出PoseFormerV2，该方法通过探索频域中长序列骨架的紧凑表示，有效扩展感受野并增强对噪声2D关节检测的鲁棒性。在仅对PoseFormer进行最小修改的前提下，所提方法实现了时域与频域特征的有效融合，相较于原始模型取得了更优的速度-精度权衡。在两个基准数据集（Human3.6M和MPI-INF-3DHP）上的大量实验表明，所提方法显著优于原始PoseFormer及其他基于Transformer的变体。代码已开源至：\url{https://github.com/QitaoZhao/PoseFormerV2}。