Reconstructing realistic animal fur geometry from images is a challenging task due to the fine-scale details, self-occlusion, and view-dependent appearance of fur. In contrast to human hairstyle reconstruction, there are also no datasets that can be leveraged to learn a fur prior for different animals. In this work, we present a first multi-view-based method for high-fidelity 3D fur modeling of animals using a strand-based representation, leveraging the general knowledge of a vision language model. Given multi-view RGB images, we first reconstruct a coarse surface geometry using traditional multi-view stereo techniques. We then use a vision language model (VLM) system to retrieve information about the realistic length structure of the fur for each part of the body. We use this knowledge to construct the animal's furless geometry and grow strands atop it. The fur reconstruction is supervised with both geometric and photometric losses computed from multi-view images. To mitigate orientation ambiguities stemming from the Gabor filters that are applied to the input images, we additionally utilize the VLM to guide the strands' growth direction and their relation to the gravity vector that we incorporate as a loss. With this new schema of using a VLM to guide 3D reconstruction from multi-view inputs, we show generalization across a variety of animals with different fur types. For additional results and code, please refer to https://neuralfur.is.tue.mpg.de.

翻译：从图像重建逼真的动物毛发几何结构是一项极具挑战性的任务，这源于毛发的精细细节、自遮挡以及视角依赖的外观特性。与人类发型重建不同，目前尚无可用于学习不同动物毛发先验的数据集。在本工作中，我们提出了一种首个基于多视角的高保真三维动物毛发建模方法，该方法采用基于发丝的表征方式，并利用了视觉语言模型的通用知识。给定多视角RGB图像，我们首先使用传统的多视角立体视觉技术重建粗糙的表面几何结构。随后，我们采用一个视觉语言模型系统来检索身体各部位真实毛发长度结构的信息。我们利用这些知识构建动物的无毛几何体，并在其表面生长发丝。毛发重建过程通过从多视角图像计算得到的几何损失和光度损失进行监督。为了缓解因对输入图像应用Gabor滤波器而产生的方向模糊性，我们还利用VLM来引导发丝的生长方向及其与重力矢量的关系，并将此作为一项损失函数纳入优化。通过这种利用VLM引导从多视角输入进行三维重建的新范式，我们展示了该方法在具有不同毛发类型的多种动物上的泛化能力。更多结果和代码，请访问 https://neuralfur.is.tue.mpg.de。