In the wake of many new ML-inspired approaches for reconstructing and representing high-quality 3D content, recent hybrid and explicitly learned representations exhibit promising performance and quality characteristics. However, their scaling to higher dimensions is challenging, e.g. when accounting for dynamic content with respect to additional parameters such as material properties, illumination, or time. In this paper, we tackle these challenges for an explicit representations based on Gaussian mixture models. With our solutions, we arrive at efficient fitting of compact N-dimensional Gaussian mixtures and enable efficient evaluation at render time: For fast fitting and evaluation, we introduce a high-dimensional culling scheme that efficiently bounds N-D Gaussians, inspired by Locality Sensitive Hashing. For adaptive refinement yet compact representation, we introduce a loss-adaptive density control scheme that incrementally guides the use of additional capacity towards missing details. With these tools we can for the first time represent complex appearance that depends on many input dimensions beyond position or viewing angle within a compact, explicit representation optimized in minutes and rendered in milliseconds.

翻译：随着众多受机器学习启发的新方法不断涌现，用于重建和表示高质量三维内容，近期的混合式与显式学习表示展现出优异的性能与质量特性。然而，这些方法向更高维度扩展时面临挑战，例如在考虑动态内容涉及材质属性、光照或时间等附加参数时。本文针对基于高斯混合模型的显式表示方法应对这些挑战。通过我们提出的解决方案，我们实现了紧凑N维高斯混合模型的高效拟合，并确保渲染时的高效计算：为实现快速拟合与计算，我们提出一种受局部敏感哈希启发的高维剔除方案，可高效界定N维高斯函数；为实现自适应细化同时保持表示紧凑性，我们引入一种损失自适应的密度控制方案，该方案能渐进地将额外容量引导至缺失细节区域。借助这些工具，我们首次能够在紧凑的显式表示中表征复杂外观——这些外观不仅取决于位置或视角，还依赖于多个输入维度，且该表示可在数分钟内完成优化，并在毫秒级时间内完成渲染。