We show how to transform a non-differentiable rasterizer into a differentiable one with minimal engineering efforts and no external dependencies (no Pytorch/Tensorflow). We rely on Stochastic Gradient Estimation, a technique that consists of rasterizing after randomly perturbing the scene's parameters such that their gradient can be stochastically estimated and descended. This method is simple and robust but does not scale in dimensionality (number of scene parameters). Our insight is that the number of parameters contributing to a given rasterized pixel is bounded. Estimating and averaging gradients on a per-pixel basis hence bounds the dimensionality of the underlying optimization problem and makes the method scalable. Furthermore, it is simple to track per-pixel contributing parameters by rasterizing ID- and UV-buffers, which are trivial additions to a rasterization engine if not already available. With these minor modifications, we obtain an in-engine optimizer for 3D assets with millions of geometry and texture parameters.

翻译：我们展示了如何以最小的工程工作量且无需外部依赖（无需PyTorch/TensorFlow）将非可微光栅化器转换为可微光栅化器。该方法基于随机梯度估计技术，即在随机扰动场景参数后执行光栅化，从而能够随机估计其梯度并进行梯度下降。该方法简单且鲁棒，但在维度（即场景参数数量）上不具备可扩展性。我们的关键洞察在于：每个光栅化像素所贡献的参数数量是有上限的。因此，基于逐像素估计和平均梯度，可有效限定底层优化问题的维度，从而使方法具有可扩展性。此外，通过光栅化ID缓冲区和UV缓冲区（若光栅化引擎尚未提供，则添加这些功能非常容易），可简便地追踪每个像素的贡献参数。借助这些微小修改，我们获得了针对包含数百万几何与纹理参数的三维资产的引擎内优化器。