Human mesh reconstruction (HMR) provides direct insights into body-environment interaction, which enables various immersive applications. While existing large-scale HMR datasets rely heavily on line-of-sight RGB input, vision-based sensing is limited by occlusion, lighting variation, and privacy concerns. To overcome these limitations, recent efforts have explored radio-frequency (RF) mmWave radar for privacy-preserving indoor human sensing. However, current radar datasets are constrained by sparse skeleton labels, limited scale, and simple in-place actions. To advance the HMR research community, we introduce M4Human, the current largest-scale (661K-frame) ($9\times$ prior largest) multimodal benchmark, featuring high-resolution mmWave radar, RGB, and depth data. M4Human provides both raw radar tensors (RT) and processed radar point clouds (RPC) to enable research across different levels of RF signal granularity. M4Human includes high-quality motion capture (MoCap) annotations with 3D meshes and global trajectories, and spans 20 subjects and 50 diverse actions, including in-place, sit-in-place, and free-space sports or rehabilitation movements. We establish benchmarks on both RT and RPC modalities, as well as multimodal fusion with RGB-D modalities. Extensive results highlight the significance of M4Human for radar-based human modeling while revealing persistent challenges under fast, unconstrained motion. The dataset and code will be released after the paper publication.

翻译：人体网格重建（HMR）为理解人体与环境交互提供了直接洞察，支撑着各类沉浸式应用。现有的大规模HMR数据集主要依赖视距内的RGB输入，而基于视觉的感知易受遮挡、光照变化及隐私问题的限制。为突破这些局限，近期研究开始探索利用射频毫米波雷达实现保护隐私的室内人体感知。然而，现有雷达数据集受限于稀疏的骨架标注、有限规模及简单的原地动作。为推动HMR研究领域发展，本文提出M4Human——当前最大规模（66.1万帧，为先前最大数据集的9倍）的多模态基准数据集，包含高分辨率毫米波雷达、RGB及深度数据。M4Human同时提供原始雷达张量（RT）与处理后的雷达点云（RPC），以支持不同粒度射频信号层面的研究。数据集配备高质量动作捕捉（MoCap）标注，包含三维网格与全局运动轨迹，涵盖20名受试者的50种多样化动作，包括原地动作、坐姿动作以及自由空间中的运动或康复动作。我们基于RT与RPC模态分别建立了基准测试，并探索了与RGB-D模态的多模态融合方法。大量实验结果验证了M4Human对雷达基人体建模研究的重要意义，同时揭示了其在快速无约束运动场景下仍存在的持续挑战。数据集与代码将在论文发表后开源。