Thermal scene reconstruction holds great potential for various applications, such as analyzing building energy consumption and performing non-destructive infrastructure testing. However, existing methods typically require dense scene measurements and often rely on RGB images for 3D geometry reconstruction, projecting thermal information post-reconstruction. This can lead to inconsistencies between the reconstructed geometry and temperature data and their actual values. To address this challenge, we propose ThermoNeRF, a novel multimodal approach based on Neural Radiance Fields that jointly renders new RGB and thermal views of a scene, and ThermoScenes, a dataset of paired RGB+thermal images comprising 8 scenes of building facades and 8 scenes of everyday objects. To address the lack of texture in thermal images, ThermoNeRF uses paired RGB and thermal images to learn scene density, while separate networks estimate color and temperature data. Unlike comparable studies, our focus is on temperature reconstruction and experimental results demonstrate that ThermoNeRF achieves an average mean absolute error of 1.13C and 0.41C for temperature estimation in buildings and other scenes, respectively, representing an improvement of over 50% compared to using concatenated RGB+thermal data as input to a standard NeRF. Code and dataset are available online.

翻译：热成像场景重建在建筑能耗分析、无损基础设施检测等应用中具有巨大潜力。然而，现有方法通常需要密集的场景测量，且多依赖RGB图像进行三维几何重建，而后再投影热信息。这种做法可能导致重建的几何结构与温度数据同实际值之间存在不一致性。为应对这一挑战，我们提出了ThermoNeRF——一种基于神经辐射场的新型多模态方法，能够联合渲染场景的RGB与热成像新视角；同时构建了ThermoScenes数据集，包含8组建筑立面场景和8组日常物体场景的配对RGB+热成像图像。针对热成像图像缺乏纹理的问题，ThermoNeRF利用配对的RGB与热成像图像学习场景密度，而通过独立网络分别估计颜色与温度数据。与同类研究不同，我们的重点在于温度重建。实验结果表明，ThermoNeRF在建筑与其他场景的温度估计中，平均绝对误差分别为1.13°C与0.41°C，相较于将拼接的RGB+热成像数据输入标准NeRF的方法，性能提升超过50%。代码与数据集已公开。