Fitting an underlying body model to 3D clothed human assets has been extensively studied, yet most approaches focus on either single-modal inputs such as point clouds or multi-view images alone, often requiring a known metric scale. This constraint is frequently impractical, especially for AI-generated assets where scale distortion is common. We propose OmniFit, a method that can seamlessly handle diverse multi-modal inputs, including full scans, partial depth observations, and image captures, while remaining scale-agnostic for both real and synthetic assets. Our key innovation is a simple yet effective conditional transformer decoder that directly maps surface points to dense body landmarks, which are then used for SMPL-X parameter fitting. In addition, an optional plug-and-play image adapter incorporates visual cues to compensate for missing geometric information. We further introduce a dedicated scale predictor that rescales subjects to canonical body proportions. OmniFit substantially outperforms state-of-the-art methods by 57.1 to 80.9 percent across daily and loose clothing scenarios. To the best of our knowledge, it is the first body fitting method to surpass multi-view optimization baselines and the first to achieve millimeter-level accuracy on the CAPE and 4D-DRESS benchmarks.

翻译：将底层人体模型与三维穿着衣物的资产进行拟合已被广泛研究，但大多数方法仅关注单模态输入（如点云或多视角图像），且通常需要已知的度量尺度。这一约束在实际应用中往往不切实际，尤其在尺度畸变普遍存在的AI生成资产场景中。我们提出OmniFit，该方法能无缝处理包括全身扫描、部分深度观测和图像采集在内的多样化多模态输入，同时对真实与合成资产的尺度保持无关性。其核心创新在于一个简洁而高效的条件Transformer解码器，可直接将表面点映射至密集人体地标，并用于SMPL-X参数拟合。此外，可选的即插即用图像适配器能整合视觉线索以补偿缺失的几何信息。我们进一步引入专用尺度预测器，将对象重新缩放到标准人体比例。OmniFit在日常与宽松衣物场景中相比现有最优方法性能提升57.1%至80.9%。据我们所知，这是首个超越多视角优化基线的人体拟合方法，也是在CAPE和4D-DRESS基准上首次达到毫米级精度的方法。