The workload of real-time rendering is steeply increasing as the demand for high resolution, high refresh rates, and high realism rises, overwhelming most graphics cards. To mitigate this problem, one of the most popular solutions is to render images at a low resolution to reduce rendering overhead, and then manage to accurately upsample the low-resolution rendered image to the target resolution, a.k.a. super-resolution techniques. Most existing methods focus on exploiting information from low-resolution inputs, such as historical frames. The absence of high frequency details in those LR inputs makes them hard to recover fine details in their high-resolution predictions. In this paper, we propose an efficient and effective super-resolution method that predicts high-quality upsampled reconstructions utilizing low-cost high-resolution auxiliary G-Buffers as additional input. With LR images and HR G-buffers as input, the network requires to align and fuse features at multi resolution levels. We introduce an efficient and effective H-Net architecture to solve this problem and significantly reduce rendering overhead without noticeable quality deterioration. Experiments show that our method is able to produce temporally consistent reconstructions in $4 \times 4$ and even challenging $8 \times 8$ upsampling cases at 4K resolution with real-time performance, with substantially improved quality and significant performance boost compared to existing works.

翻译：实时渲染的工作量随着对高分辨率、高帧率和高真实感需求的增加而急剧上升，使大多数图形卡不堪重负。为缓解这一问题，一种流行的解决方案是以低分辨率渲染图像以降低渲染开销，随后设法将低分辨率渲染图像精确上采样至目标分辨率，即超分辨率技术。现有方法大多侧重于利用低分辨率输入（如历史帧）中的信息。这些低分辨率输入中高频细节的缺失，使其难以恢复高分辨率预测中的精细细节。本文提出一种高效且有效的超分辨率方法，利用低成本的辅助高分辨率G-Buffer作为额外输入，预测高质量的上采样重构结果。以低分辨率图像和高分辨率G-Buffer为输入，网络需要对齐并融合多分辨率级别的特征。我们引入一种高效且有效的H-Net架构来解决该问题，在无明显质量退化的情况下显著降低渲染开销。实验表明，所提方法能够在4K分辨率下实现4×4乃至具有挑战性的8×8上采样情况下生成时间一致的重构结果，并具备实时性能，与现有工作相比质量大幅提升，性能显著增强。