The Differentiable Rendering and Implicit Function-based model (DRIFu) draws its roots from the Pixel-aligned Implicit Function (PIFU), a pioneering 3D digitization technique initially designed for clothed human bodies. PIFU excels in capturing nuanced body shape variations within a low-dimensional space and has been extensively trained on human 3D scans. However, the application of PIFU to live animals poses significant challenges, primarily due to the inherent difficulty in obtaining the cooperation of animals for 3D scanning. In response to this challenge, we introduce the DRIFu model, specifically tailored for animal digitization. To train DRIFu, we employ a curated set of synthetic 3D animal models, encompassing diverse shapes, sizes, and even accounting for variations such as baby birds. Our innovative alignment tools play a pivotal role in mapping these diverse synthetic animal models onto a unified template, facilitating precise predictions of animal shape and texture. Crucially, our template alignment strategy establishes a shared shape space, allowing for the seamless sampling of new animal shapes, posing them realistically, animating them, and aligning them with real-world data. This groundbreaking approach revolutionizes our capacity to comprehensively understand and represent avian forms. For further details and access to the project, the project website can be found at https://github.com/kuangzijian/drifu-for-animals

翻译：可微渲染与隐函数模型（DRIFu）源自像素对齐隐函数（PIFU），后者最初是为 clothed 人体设计的开创性三维数字化技术。PIFU 在低维空间中擅长捕捉细微的体形变化，并已基于人体三维扫描数据进行了大量训练。然而，将 PIFU 应用于活体动物面临重大挑战，主要原因是难以获取动物在三维扫描中的配合。针对这一问题，我们提出了专门用于动物数字化的 DRIFu 模型。为了训练 DRIFu，我们采用了一套精心挑选的合成三维动物模型，涵盖不同的形状、大小，甚至包括雏鸟等变异形态。我们创新的对齐工具在将这些多样化的合成动物模型映射到统一模板上发挥了关键作用，从而促进了动物形状和纹理的精确预测。至关重要的是，我们的模板对齐策略建立了一个共享的形状空间，使得能够无缝采样新的动物形状、以逼真方式摆姿势、制作动画并将其与现实数据对齐。这一突破性方法彻底改变了我们全面理解和表征鸟类形态的能力。更多详情及项目资源，请访问项目网站：https://github.com/kuangzijian/drifu-for-animals