In this paper, we develop a modular neural network for real-time semantic mapping in uncertain environments, which explicitly updates per-voxel probabilistic distributions within a neural network layer. Our approach combines the reliability of classical probabilistic algorithms with the performance and efficiency of modern neural networks. Although robotic perception is often divided between modern differentiable methods and classical explicit methods, a union of both is necessary for real-time and trustworthy performance. We introduce a novel Convolutional Bayesian Kernel Inference (ConvBKI) layer which incorporates semantic segmentation predictions online into a 3D map through a depthwise convolution layer by leveraging conjugate priors. We compare ConvBKI against state-of-the-art deep learning approaches and probabilistic algorithms for mapping to evaluate reliability and performance. We also create a Robot Operating System (ROS) package of ConvBKI and test it on real-world perceptually challenging off-road driving data.

翻译：本文开发了一种模块化神经网络，用于不确定环境下的实时语义建图，该网络在神经网络层内显式更新每个体素的概率分布。我们的方法将经典概率算法的可靠性与现代神经网络的性能及效率相结合。尽管机器人感知常被划分为现代可微方法与经典显式方法，但实现实时且可信的性能需要两者的统一。我们提出了一种新颖的卷积贝叶斯核推理（ConvBKI）层，该层通过利用共轭先验，以深度卷积层的形式将语义分割预测在线整合到三维地图中。我们将ConvBKI与最先进的深度学习方法及概率建图算法进行对比，以评估其可靠性与性能。此外，我们还创建了ConvBKI的机器人操作系统（ROS）软件包，并在真实世界中具有感知挑战性的越野驾驶数据上进行了测试。