Spiking neural networks (SNNs) have been thriving on numerous tasks to leverage their promising energy efficiency and exploit their potentialities as biologically plausible intelligence. Meanwhile, the Neural Radiance Fields (NeRF) render high-quality 3D scenes with massive energy consumption, but few works delve into the energy-saving solution with a bio-inspired approach. In this paper, we propose SpikingNeRF, which aligns the radiance ray with the temporal dimension of SNN, to naturally accommodate the SNN to the reconstruction of Radiance Fields. Thus, the computation turns into a spike-based, multiplication-free manner, reducing the energy consumption. In SpikingNeRF, each sampled point on the ray is matched onto a particular time step, and represented in a hybrid manner where the voxel grids are maintained as well. Based on the voxel grids, sampled points are determined whether to be masked for better training and inference. However, this operation also incurs irregular temporal length. We propose the temporal padding strategy to tackle the masked samples to maintain regular temporal length, i.e., regular tensors, and the temporal condensing strategy to form a denser data structure for hardware-friendly computation. Extensive experiments on various datasets demonstrate that our method reduces the 70.79% energy consumption on average and obtains comparable synthesis quality with the ANN baseline.

翻译：脉冲神经网络（SNN）已在众多任务中蓬勃发展，以发挥其极具前景的能效优势并挖掘其作为生物合理智能的潜力。与此同时，神经辐射场（NeRF）以高能耗实现高质量三维场景重建，但鲜有研究探索采用生物启发式方法的节能方案。本文提出SpikingNeRF，通过将辐射光线与SNN的时间维度对齐，使SNN自然适配于辐射场的重建。由此，计算转变为基于脉冲、免乘法的形式，降低了能耗。在SpikingNeRF中，光线上的每个采样点被映射到特定时间步，并以混合方式表示，同时保留体素网格。基于体素网格，可判断采样点是否需要掩码以实现更优的训练与推理。然而，这一操作会引入不规则的时间长度。我们提出时间填充策略以处理掩码样本，维持规则的时间长度（即规则张量），并提出时间凝聚策略以形成更密集的数据结构，便于硬件高效计算。在多种数据集上的广泛实验表明，我们的方法平均降低70.79%的能耗，并实现与ANN基线相当的合成质量。