Human modeling and relighting are two fundamental problems in computer vision and graphics, where high-quality datasets can largely facilitate related research. However, most existing human datasets only provide multi-view human images captured under the same illumination. Although valuable for modeling tasks, they are not readily used in relighting problems. To promote research in both fields, in this paper, we present UltraStage, a new 3D human dataset that contains more than 2,000 high-quality human assets captured under both multi-view and multi-illumination settings. Specifically, for each example, we provide 32 surrounding views illuminated with one white light and two gradient illuminations. In addition to regular multi-view images, gradient illuminations help recover detailed surface normal and spatially-varying material maps, enabling various relighting applications. Inspired by recent advances in neural representation, we further interpret each example into a neural human asset which allows novel view synthesis under arbitrary lighting conditions. We show our neural human assets can achieve extremely high capture performance and are capable of representing fine details such as facial wrinkles and cloth folds. We also validate UltraStage in single image relighting tasks, training neural networks with virtual relighted data from neural assets and demonstrating realistic rendering improvements over prior arts. UltraStage will be publicly available to the community to stimulate significant future developments in various human modeling and rendering tasks. The dataset is available at https://miaoing.github.io/RNHA.

翻译：人体建模与重光照是计算机视觉与图形学领域的两个基础问题，高质量数据集能极大促进相关研究。然而，现有大多数人体验集仅提供在相同光照条件下采集的多视角人体图像。虽然这些数据对建模任务具有重要价值，却难以直接用于重光照问题。为促进这两个领域的研究，本文提出UltraStage——一个包含超过2000个高质量人体资产的全新3D人体数据集，该数据集在多视角与多光照条件下采集。具体而言，每个样本提供32个环绕视角的图片，分别在一个白光和两种梯度光照下拍摄。除常规多视角图像外，梯度光照有助于恢复详细的表面法向和空间变化材质贴图，从而支持多种重光照应用。受神经表征领域最新进展启发，我们将每个样本进一步解析为可重光照神经人体资产，该资产支持任意光照条件下的新视角合成。实验表明，我们的神经人体资产能实现极高的采集性能，并具备表示面部皱纹和衣物褶皱等精细细节的能力。我们还通过单图像重光照任务验证了UltraStage：利用神经资产生成的虚拟重光照数据训练神经网络，相较于现有方法实现了更逼真的渲染效果。UltraStage将向社区公开，以推动人体建模与渲染任务的未来重要发展。数据集访问链接：https://miaoing.github.io/RNHA。