Neural materials typically consist of a collection of neural features along with a decoder network. The main challenge in integrating such models in real-time rendering pipelines lies in the large size required to store their features in GPU memory and the complexity of evaluating the network efficiently. We present a neural material model whose features and decoder are specifically designed to be used in real-time rendering pipelines. Our framework leverages hardware-based block compression (BC) texture formats to store the learned features and trains the model to output the material information continuously in space and scale. To achieve this, we organize the features in a block-based manner and emulate BC6 decompression during training, making it possible to export them as regular BC6 textures. This structure allows us to use high resolution features while maintaining a low memory footprint. Consequently, this enhances our model's overall capability, enabling the use of a lightweight and simple decoder architecture that can be evaluated directly in a shader. Furthermore, since the learned features can be decoded continuously, it allows for random uv sampling and smooth transition between scales without needing any subsequent filtering. As a result, our neural material has a small memory footprint, can be decoded extremely fast adding a minimal computational overhead to the rendering pipeline.

翻译：神经材质通常由一组神经特征和解码器网络组成。将此类模型集成到实时渲染管线的主要挑战在于：在GPU内存中存储其特征所需的大量空间，以及高效评估网络的复杂性。我们提出了一种神经材质模型，其特征和解码器专为实时渲染管线而设计。我们的框架利用基于硬件的块压缩（BC）纹理格式来存储学习到的特征，并训练模型在空间和尺度上连续输出材质信息。为实现这一点，我们以块为基础组织特征，并在训练过程中模拟BC6解压缩，从而能够将其导出为常规BC6纹理。这种结构使我们能够使用高分辨率特征，同时保持较低的内存占用。因此，这增强了模型的整体能力，使得能够采用可直接在着色器中评估的轻量级简单解码器架构。此外，由于学习到的特征可以连续解码，它允许随机UV采样和不同尺度间的平滑过渡，无需后续滤波处理。因此，我们的神经材质具有较小的内存占用，解码速度极快，仅为渲染管线增加最小的计算开销。