The ability to accurately represent and localise relevant objects is essential for robots to carry out tasks effectively. Traditional approaches, where robots simply capture an image, process that image to take an action, and then forget the information, have proven to struggle in the presence of occlusions. Methods using multi-view perception, which have the potential to address some of these problems, require a world model that guides the collection, integration and extraction of information from multiple viewpoints. Furthermore, constructing a generic representation that can be applied in various environments and tasks is a difficult challenge. In this paper, a novel approach for building generic representations in occluded agro-food environments using multi-view perception and 3D multi-object tracking is introduced. The method is based on a detection algorithm that generates partial point clouds for each detected object, followed by a 3D multi-object tracking algorithm that updates the representation over time. The accuracy of the representation was evaluated in a real-world environment, where successful representation and localisation of tomatoes in tomato plants were achieved, despite high levels of occlusion, with the total count of tomatoes estimated with a maximum error of 5.08% and the tomatoes tracked with an accuracy up to 71.47%. Novel tracking metrics were introduced, demonstrating that valuable insight into the errors in localising and representing the fruits can be provided by their use. This approach presents a novel solution for building representations in occluded agro-food environments, demonstrating potential to enable robots to perform tasks effectively in these challenging environments.

翻译：准确表征和定位相关目标物体是机器人有效执行任务的关键。传统方法中，机器人仅通过拍摄单张图像、处理该图像并执行相应动作后即丢弃信息，此类方法在处理遮挡问题时效果有限。采用多视角感知的方法虽有望解决部分问题，但需要构建一个能指导多视角信息采集、整合与提取的世界模型。此外，构建可适用于不同环境和任务的通用表征方式仍是一项艰巨挑战。本文提出了一种利用多视角感知与3D多目标追踪在遮挡型农业食品环境中构建通用表征的新方法。该方法基于检测算法为每个检测对象生成局部点云，随后通过3D多目标追踪算法随时间动态更新表征。在真实环境中对该表征的准确性进行了评估，尽管存在严重遮挡，仍成功实现了番茄植株果实的高精度表征与定位：果实总数估算最大误差为5.08%，果实追踪准确率高达71.47%。本研究引入新型追踪评价指标，证明其能为定位和表征果实过程中的误差分析提供有价值的信息。该方案为在遮挡型农业食品环境中构建表征提供了创新解决方案，展示了机器人在此类挑战性环境下有效执行任务的潜力。