We show how to transform a non-differentiable rasterizer into a differentiable one with minimal engineering efforts and no external dependencies (no Pytorch/Tensorflow). We rely on Stochastic Gradient Estimation, a technique that consists of rasterizing after randomly perturbing the scene's parameters such that their gradient can be stochastically estimated and descended. This method is simple and robust but does not scale in dimensionality (number of scene parameters). Our insight is that the number of parameters contributing to a given rasterized pixel is bounded. Estimating and averaging gradients on a per-pixel basis hence bounds the dimensionality of the underlying optimization problem and makes the method scalable. Furthermore, it is simple to track per-pixel contributing parameters by rasterizing ID- and UV-buffers, which are trivial additions to a rasterization engine if not already available. With these minor modifications, we obtain an in-engine optimizer for 3D assets with millions of geometry and texture parameters.

翻译：我们展示了如何以最小的工程开销且无需外部依赖（无需PyTorch/TensorFlow）将非可微分光栅化器转换为可微分光栅化器。我们采用随机梯度估计技术，该技术通过在随机扰动场景参数后进行光栅化，使得参数梯度能够被随机估计并用于梯度下降。该方法简单且鲁棒，但在参数维度（场景参数数量）上存在扩展性限制。我们的关键洞察在于：每个光栅化像素所贡献的参数数量是有界的。通过逐像素估计和平均梯度，优化问题的底层维度得到约束，从而使该方法具备扩展性。此外，通过光栅化ID缓冲区和UV缓冲区（若光栅化引擎未提供这些功能，则可轻易添加），可以简单追踪每个像素的贡献参数。借助这些轻微修改，我们即可获得一个用于优化包含数百万几何与纹理参数的3D资产的引擎内优化器。