Scene representation networks (SRNs) have been recently proposed for compression and visualization of scientific data. However, state-of-the-art SRNs do not adapt the allocation of available network parameters to the complex features found in scientific data, leading to a loss in reconstruction quality. We address this shortcoming with an adaptively placed multi-grid SRN (APMGSRN) and propose a domain decomposition training and inference technique for accelerated parallel training on multi-GPU systems. We also release an open-source neural volume rendering application that allows plug-and-play rendering with any PyTorch-based SRN. Our proposed APMGSRN architecture uses multiple spatially adaptive feature grids that learn where to be placed within the domain to dynamically allocate more neural network resources where error is high in the volume, improving state-of-the-art reconstruction accuracy of SRNs for scientific data without requiring expensive octree refining, pruning, and traversal like previous adaptive models. In our domain decomposition approach for representing large-scale data, we train an set of APMGSRNs in parallel on separate bricks of the volume to reduce training time while avoiding overhead necessary for an out-of-core solution for volumes too large to fit in GPU memory. After training, the lightweight SRNs are used for realtime neural volume rendering in our open-source renderer, where arbitrary view angles and transfer functions can be explored. A copy of this paper, all code, all models used in our experiments, and all supplemental materials and videos are available at https://github.com/skywolf829/APMGSRN.
翻译:场景表示网络(SRN)近期被提出用于科学数据的压缩与可视化。然而,最先进的SRN未能将可用网络参数的分配适应于科学数据中的复杂特征,导致重建质量下降。我们通过自适应放置的多网格SRN(APMGSRN)解决了这一不足,并提出了一种域分解训练和推理技术,用于在多GPU系统上加速并行训练。我们还发布了一款开源神经体渲染应用程序,支持与任何基于PyTorch的SRN进行即插即用式渲染。我们提出的APMGSRN架构利用多个空间自适应特征网格,学习在域内何处放置这些网格,从而动态地将更多神经网络资源分配到体积中误差较高的区域,提升了SRN在科学数据上的最先进重建精度,且无需像以往的适应模型那样进行昂贵的八叉树细化、剪枝和遍历。在表示大规模数据时,我们采用域分解方法并行训练一组APMGSRN,分别作用于体积的不同块,以减少训练时间,同时避免针对超出GPU内存容量的大体积数据所需的核外解决方案带来的开销。训练后,轻量级SRN被用于我们的开源渲染器中的实时神经体渲染,用户可探索任意视角和传递函数。本文、所有代码、实验中使用的所有模型以及所有补充材料和视频均可从 https://github.com/skywolf829/APMGSRN 获取。