We present Neural Memory Object (NeMO), a novel object-centric representation that can be used to detect, segment and estimate the 6DoF pose of objects unseen during training using RGB images. Our method consists of an encoder that requires only a few RGB template views depicting an object to generate a sparse object-like point cloud using a learned UDF containing semantic and geometric information. Next, a decoder takes the object encoding together with a query image to generate a variety of dense predictions. Through extensive experiments, we show that our method can be used for few-shot object perception without requiring any camera-specific parameters or retraining on target data. Our proposed concept of outsourcing object information in a NeMO and using a single network for multiple perception tasks enhances interaction with novel objects, improving scalability and efficiency by enabling quick object onboarding without retraining or extensive pre-processing. We report competitive and state-of-the-art results on various datasets and perception tasks of the BOP benchmark, demonstrating the versatility of our approach. https://github.com/DLR-RM/nemo

翻译：我们提出神经记忆对象（NeMO），这是一种新颖的以对象为中心的表示方法，可用于通过RGB图像检测、分割并估计训练期间未见过的物体的6自由度姿态。我们的方法包含一个编码器，该编码器仅需少量描绘物体的RGB模板视图，即可利用包含语义和几何信息的已学习UDF生成稀疏的类物体点云。随后，解码器将对象编码与查询图像结合，生成多种密集预测。通过大量实验，我们证明该方法可用于少样本物体感知，且无需任何相机特定参数或在目标数据上重新训练。我们提出的将物体信息外包至NeMO并使用单一网络处理多种感知任务的概念，增强了与新颖物体的交互能力，通过实现无需重新训练或大量预处理的快速物体接入，提升了可扩展性和效率。我们在BOP基准测试的多个数据集和感知任务上报告了具有竞争力且达到最先进水平的结果，证明了我们方法的通用性。https://github.com/DLR-RM/nemo