Clothing is an important part of human appearance but challenging to model in photorealistic avatars. In this work we present avatars with dynamically moving loose clothing that can be faithfully driven by sparse RGB-D inputs as well as body and face motion. We propose a Neural Iterative Closest Point (N-ICP) algorithm that can efficiently track the coarse garment shape given sparse depth input. Given the coarse tracking results, the input RGB-D images are then remapped to texel-aligned features, which are fed into the drivable avatar models to faithfully reconstruct appearance details. We evaluate our method against recent image-driven synthesis baselines, and conduct a comprehensive analysis of the N-ICP algorithm. We demonstrate that our method can generalize to a novel testing environment, while preserving the ability to produce high-fidelity and faithful clothing dynamics and appearance.

翻译：服装是人类外观的重要组成部分，但在逼真的虚拟形象中建模极具挑战性。本文提出一种支持动态松散服装的虚拟形象，能够通过稀疏RGB-D输入以及身体和面部运动进行可信驱动。我们提出神经迭代最近点（N-ICP）算法，可在稀疏深度输入下高效跟踪粗略服装形状。基于粗跟踪结果，输入RGB-D图像被重新映射到纹理对齐特征，并输入可驾驶虚拟形象模型以忠实重建外观细节。我们将该方法与近期基于图像驱动的合成基线进行对比评估，并对N-ICP算法进行综合分析。实验证明，该方法能够泛化至新颖测试环境，同时保持生成高保真、可信的服装动态与外观的能力。