The Differentiable Rendering and Implicit Function-based model (DRIFu) draws its roots from the Pixel-aligned Implicit Function (PIFU), a pioneering 3D digitization technique initially designed for clothed human bodies. PIFU excels in capturing nuanced body shape variations within a low-dimensional space and has been extensively trained on human 3D scans. However, the application of PIFU to live animals poses significant challenges, primarily due to the inherent difficulty in obtaining the cooperation of animals for 3D scanning. In response to this challenge, we introduce the DRIFu model, specifically tailored for animal digitization. To train DRIFu, we employ a curated set of synthetic 3D animal models, encompassing diverse shapes, sizes, and even accounting for variations such as baby birds. Our innovative alignment tools play a pivotal role in mapping these diverse synthetic animal models onto a unified template, facilitating precise predictions of animal shape and texture. Crucially, our template alignment strategy establishes a shared shape space, allowing for the seamless sampling of new animal shapes, posing them realistically, animating them, and aligning them with real-world data. This groundbreaking approach revolutionizes our capacity to comprehensively understand and represent avian forms. For further details and access to the project, the project website can be found at https://github.com/kuangzijian/drifu-for-animals

翻译：可微渲染与隐式函数模型（DRIFu）源于像素对齐隐式函数（PIFU），后者最初是为穿衣人体设计的开创性三维数字化技术。PIFU擅长在低维空间中捕捉细微的人体形态变化，并已基于大量人体三维扫描数据进行了训练。然而，将PIFU应用于活体动物面临显著挑战，主要原因在于获取动物配合进行三维扫描存在固有困难。针对这一挑战，我们提出了专为动物数字化定制的DRIFu模型。为训练DRIFu，我们采用了一组精心筛选的合成三维动物模型，涵盖不同形状、尺寸，甚至包括雏鸟等形态变异。我们创新的对齐工具在将这些多样的合成动物模型映射到统一模板方面发挥了关键作用，从而能够精确预测动物形状与纹理。至关重要的是，我们的模板对齐策略建立了一个共享形状空间，使得新动物形状的无缝采样成为可能，并能够以真实方式对其进行姿态设置、动画制作以及与真实世界数据对齐。这一突破性方法彻底革新了我们全面理解并表征鸟类形态的能力。更多详情及项目访问，请参见项目网站：https://github.com/kuangzijian/drifu-for-animals