Differentiable rendering is a technique used in an important emerging class of visual computing applications that involves representing a 3D scene as a model that is trained from 2D images using gradient descent. Recent works (e.g. 3D Gaussian Splatting) use a rasterization pipeline to enable rendering high quality photo-realistic imagery at high speeds from these learned 3D models. These methods have been demonstrated to be very promising, providing state-of-art quality for many important tasks. However, training a model to represent a scene is still a time-consuming task even when using powerful GPUs. In this work, we observe that the gradient computation phase during training is a significant bottleneck on GPUs due to the large number of atomic operations that need to be processed. These atomic operations overwhelm atomic units in the L2 partitions causing stalls. To address this challenge, we leverage the observations that during the gradient computation: (1) for most warps, all threads atomically update the same memory locations; and (2) warps generate varying amounts of atomic traffic (since some threads may be inactive). We propose DISTWAR, a software-approach to accelerate atomic operations based on two key ideas: First, we enable warp-level reduction of threads at the SM sub-cores using registers to leverage the locality in intra-warp atomic updates. Second, we distribute the atomic computation between the warp-level reduction at the SM and the L2 atomic units to increase the throughput of atomic computation. Warps with many threads performing atomic updates to the same memory locations are scheduled at the SM, and the rest using L2 atomic units. We implement DISTWAR using existing warp-level primitives. We evaluate DISTWAR on widely used raster-based differentiable rendering workloads. We demonstrate significant speedups of 2.44x on average (up to 5.7x).

翻译：可微渲染是一种用于新兴视觉计算应用的重要技术，它通过梯度下降从二维图像中训练表示三维场景的模型。近期研究（如3D高斯泼溅法）利用光栅化管线，从这些学习到的三维模型中快速渲染高质量、照片级逼真的图像。这些方法已被证明极具潜力，为许多重要任务提供了最先进的性能。然而，即使在强大的GPU上，训练表示场景的模型仍是一项耗时的任务。在本工作中，我们观察到训练中的梯度计算阶段由于需要处理大量原子操作，成为GPU上的显著瓶颈，这些原子操作使L2分区中的原子单元过载并导致停滞。为解决这一挑战，我们利用梯度计算中的两个观察结果：(1) 大多数线程束中，所有线程以原子方式更新相同的内存位置；(2) 不同线程束产生的原子流量差异较大（因部分线程可能处于非活跃状态）。我们提出DISTWAR，这是一种基于两个核心思想加速原子操作的软件方案：首先，我们在SM子核上利用寄存器实现线程束级归约，以利用线程束内原子更新的局部性；其次，我们将原子计算分布在SM的线程束级归约与L2原子单元之间，以提升原子计算的吞吐量。具有大量线程对相同内存位置执行原子更新的线程束在SM上调度，其余线程束使用L2原子单元。我们利用现有的线程束级原语实现了DISTWAR，并在广泛使用的基于光栅化的可微渲染负载上进行了评估。实验表明，我们实现了平均2.44倍（最高5.7倍）的显著加速。