In this paper, we develop a modular neural network for real-time semantic mapping in uncertain environments, which explicitly updates per-voxel probabilistic distributions within a neural network layer. Our approach combines the reliability of classical probabilistic algorithms with the performance and efficiency of modern neural networks. Although robotic perception is often divided between modern differentiable methods and classical explicit methods, a union of both is necessary for real-time and trustworthy performance. We introduce a novel Convolutional Bayesian Kernel Inference (ConvBKI) layer which incorporates semantic segmentation predictions online into a 3D map through a depthwise convolution layer by leveraging conjugate priors. We compare ConvBKI against state-of-the-art deep learning approaches and probabilistic algorithms for mapping to evaluate reliability and performance. We also create a Robot Operating System (ROS) package of ConvBKI and test it on real-world perceptually challenging off-road driving data.

翻译：本文提出了一种模块化神经网络，用于在不确定环境中进行实时语义建图，该网络在神经网络层内显式更新每个体素的概率分布。我们的方法结合了经典概率算法的可靠性与现代神经网络的性能及效率。尽管机器人感知常被划分为现代可微方法与经典显式方法，但两者的结合对于实现实时且可信的性能至关重要。我们引入了一种新型卷积贝叶斯核推理（ConvBKI）层，该层通过利用共轭先验，以深度卷积层的形式将语义分割预测在线集成至三维地图中。我们将ConvBKI与最先进的深度学习方法及概率建图算法进行对比，以评估其可靠性与性能。同时，我们创建了ConvBKI的机器人操作系统（ROS）软件包，并在真实世界中具有感知挑战性的越野驾驶数据上进行了测试。