Prior work for articulated 3D shape reconstruction often relies on specialized sensors (e.g., synchronized multi-camera systems), or pre-built 3D deformable models (e.g., SMAL or SMPL). Such methods are not able to scale to diverse sets of objects in the wild. We present BANMo, a method that requires neither a specialized sensor nor a pre-defined template shape. BANMo builds high-fidelity, articulated 3D models (including shape and animatable skinning weights) from many monocular casual videos in a differentiable rendering framework. While the use of many videos provides more coverage of camera views and object articulations, they introduce significant challenges in establishing correspondence across scenes with different backgrounds, illumination conditions, etc. Our key insight is to merge three schools of thought; (1) classic deformable shape models that make use of articulated bones and blend skinning, (2) volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and (3) canonical embeddings that generate correspondences between pixels and an articulated model. We introduce neural blend skinning models that allow for differentiable and invertible articulated deformations. When combined with canonical embeddings, such models allow us to establish dense correspondences across videos that can be self-supervised with cycle consistency. On real and synthetic datasets, BANMo shows higher-fidelity 3D reconstructions than prior works for humans and animals, with the ability to render realistic images from novel viewpoints and poses. Project webpage: banmo-www.github.io .

翻译：先前针对铰接式三维形状重建的工作通常依赖于专用传感器（例如同步多相机系统）或预构建的三维可变形模型（例如SMAL或SMPL）。此类方法无法扩展到野外多样化的物体类别。我们提出BANMo，一种既不需要专用传感器也不需要预定义模板形状的方法。BANMo在可微渲染框架中，从大量单目随意视频构建高保真、可铰接的三维模型（包括形状和可动画化的蒙皮权重）。虽然使用大量视频提供了更全面的相机视角和物体关节运动覆盖，但也引入了显著挑战——需要建立跨场景（具有不同背景、光照条件等）的对应关系。我们的关键洞察在于融合三种技术路线：（1）利用铰接骨骼和混合蒙皮的经典可变形形状模型，（2）适用于梯度优化的体素神经辐射场（NeRF），以及（3）生成像素与铰接模型间对应关系的规范嵌入。我们引入可微且可逆的铰接变形神经混合蒙皮模型。当与规范嵌入结合时，这种模型使我们能够建立跨视频的密集对应关系，并通过循环一致性实现自监督。在真实和合成数据集上，BANMo展示了比先前方法更高保真度的人体和动物三维重建，且能够从新视角和新姿态渲染逼真图像。项目页面：banmo-www.github.io。