We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations. Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior -- transformation of directions into learned shading frames -- facilitates accurate reconstruction of mesoscale effects. The second prior -- a microfacet sampling distribution -- allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation. By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.

翻译：我们提出了一套完整的实时渲染系统，能够呈现此前仅限于离线使用的复杂外观场景，这得益于算法与系统层面的双重创新。该外观模型采用经神经解码器解释的学习型层次纹理，可生成反射率与重要性采样方向。为充分挖掘解码器的建模能力，我们为其配备了两类图形学先验：第一类先验——将方向变换为学习型着色坐标系——有助于精确重建中尺度效应；第二类先验——微面片采样分布——使神经解码器能高效执行重要性采样。生成的外观模型支持各向异性采样与层次细节渲染，并可将深度分层的材质图压缩为紧凑的统一神经表征。通过向光线追踪着色器暴露硬件加速的张量运算，我们证明了在实时路径追踪器内联并高效执行神经解码器的可行性。我们分析了随神经材质数量增长的扩展性，并提出利用面向一致性与分歧性执行优化的代码来提升性能。我们的神经材质着色器比非神经分层材质着色器快逾一个数量级，这为游戏与实时预览等实时应用使用电影级视觉效果开辟了道路。