We propose an approach to 3D reconstruction via inverse procedural modeling and investigate two variants of this approach. The first option consists in the fitting set of input parameters using a genetic algorithm. We demonstrate the results of our work on tree models, complex objects, with the reconstruction of which most existing methods cannot handle. The second option allows us to significantly improve the precision by using gradients within memetic algorithm, differentiable rendering and also differentiable procedural generators. In our work we see 2 main contributions. First, we propose a method to join differentiable rendering and inverse procedural modeling. This gives us an opportunity to reconstruct 3D model more accurately than existing approaches when a small number of input images are available (even for single image). Second, we join both differentiable and non-differentiable procedural generators in a single framework which allow us to apply inverse procedural modeling to fairly complex generators: when gradient is available, reconstructions is precise, when gradient is not available, reconstruction is approximate, but always high quality without visual artifacts.
翻译:我们提出了一种基于逆过程建模的三维重建方法,并研究了该方法的两种变体。第一种方案采用遗传算法对输入参数集进行拟合。我们在树模型及复杂物体上展示了实验结果,这些物体的重建是大多数现有方法难以处理的。第二种方案通过将梯度引入模因算法、可微分渲染以及可微分过程生成器,显著提升了重建精度。本工作包含两项主要贡献:首先,我们提出了一种将可微分渲染与逆过程建模相结合的方法。这使得在输入图像数量较少(甚至仅单张图像)时,能够比现有方法更精确地重建三维模型。其次,我们将可微分与不可微分过程生成器统一整合到单个框架中,从而能够将逆过程建模应用于相当复杂的生成器;当梯度可用时,重建结果精确;当梯度不可用时,重建结果虽为近似值,但始终保持高质量且无视觉伪影。