We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior -- transformation of directions into learned shading frames -- facilitates accurate reconstruction of mesoscale effects. The second prior -- a microfacet sampling distribution -- allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.

翻译：我们提出了一套完整的系统，用于实时渲染具有复杂外观的场景，这类场景以往仅能通过离线渲染实现。这一成果是通过算法层面与系统层面的创新相结合而达成的。我们的外观模型采用经过学习的层次化纹理，这些纹理通过神经解码器进行解析，从而生成反射率值及重要性采样方向。为充分利用解码器的建模能力，我们为解码器配备了两种图形学先验知识。第一种先验——将方向转换至学习得到的着色坐标系——有助于精确重建中尺度效应。第二种先验——微表面采样分布——使得神经解码器能够高效执行重要性采样。所得的外观模型支持各向异性采样与细节层次渲染，并能够将深度分层的材质图烘焙为紧凑统一的神经表示。通过将硬件加速的张量运算引入光线追踪着色器，我们证明了在实时路径追踪器中内联并高效执行神经解码器是可行的。我们分析了随着神经材质数量增加时的可扩展性，并提出了通过针对相干与发散执行优化的代码来提升性能。我们的神经材质着色器可比非神经的分层材质快一个数量级以上。这为在实时应用（如游戏与实时预览）中使用电影级视觉质量打开了大门。