To navigate complex environments, robots must increasingly use high-dimensional visual feedback (e.g. images) for control. However, relying on high-dimensional image data to make control decisions raises important questions; particularly, how might we prove the safety of a visual-feedback controller? Control barrier functions (CBFs) are powerful tools for certifying the safety of feedback controllers in the state-feedback setting, but CBFs have traditionally been poorly-suited to visual feedback control due to the need to predict future observations in order to evaluate the barrier function. In this work, we solve this issue by leveraging recent advances in neural radiance fields (NeRFs), which learn implicit representations of 3D scenes and can render images from previously-unseen camera perspectives, to provide single-step visual foresight for a CBF-based controller. This novel combination is able to filter out unsafe actions and intervene to preserve safety. We demonstrate the effect of our controller in real-time simulation experiments where it successfully prevents the robot from taking dangerous actions.

翻译：为在复杂环境中导航，机器人需越来越多地依赖高维视觉反馈（如图像）进行控制。然而，基于高维图像数据制定控制决策引发重要问题：如何证明视觉反馈控制器的安全性？控制障碍函数（CBF）是状态反馈场景下认证反馈控制器安全性的有力工具，但传统上因需预测未来观测值以评估障碍函数，难以适配视觉反馈控制。本研究通过利用神经辐射场（NeRF）的最新进展解决该问题——NeRF能学习三维场景的隐式表征，并从未见过的相机视角渲染图像，从而为基于CBF的控制器提供单步视觉前瞻能力。这种创新组合可过滤不安全动作并进行干预以维护安全性。我们通过实时仿真实验验证了该控制器的效果，成功阻止了机器人采取危险行为。