Object-centric representations enable autonomous driving algorithms to reason about interactions between many independent agents and scene features. Traditionally these representations have been obtained via supervised learning, but this decouples perception from the downstream driving task and could harm generalization. In this work we adapt a self-supervised object-centric vision model to perform object decomposition using only RGB video and the pose of the vehicle as inputs. We demonstrate that our method obtains promising results on the Waymo Open perception dataset. While object mask quality lags behind supervised methods or alternatives that use more privileged information, we find that our model is capable of learning a representation that fuses multiple camera viewpoints over time and successfully tracks many vehicles and pedestrians in the dataset. Code for our model is available at https://github.com/wayveai/SOCS.

翻译：面向自监督以对象为中心的感知：连接视觉与运动 基于对象的表征使自动驾驶算法能够推理多个独立智能体与场景特征之间的交互。传统上，这些表征通过监督学习获得，但这种方式将感知与下游驾驶任务解耦，可能损害泛化能力。本研究将自监督以对象为中心的视觉模型进行适配，仅以RGB视频和车辆位姿作为输入，实现对场景对象的分解。实验表明，该方法在Waymo开放感知数据集上取得了有前景的结果。尽管对象掩膜质量落后于使用更多特权信息的监督方法或替代方案，我们发现该模型能够学习一种随时间融合多摄像头视角的表征，并成功追踪数据集中的多辆车辆和行人。模型代码发布于https://github.com/wayveai/SOCS。