Following crop growth through the vegetative cycle allows farmers to predict fruit setting and yield in early stages, but it is a laborious and non-scalable task if performed by a human who has to manually measure fruit sizes with a caliper or dendrometers. In recent years, computer vision has been used to automate several tasks in precision agriculture, such as detecting and counting fruits, and estimating their size. However, the fundamental problem of matching the exact same fruits from one video, collected on a given date, to the fruits visible in another video, collected on a later date, which is needed to track fruits' growth through time, remains to be solved. Few attempts were made, but they either assume that the camera always starts from the same known position and that there are sufficiently distinct features to match, or they used other sources of data like GPS. Here we propose a new paradigm to tackle this problem, based on constellations of 3D centroids, and introduce a descriptor for very sparse 3D point clouds that can be used to match fruits across videos. Matching constellations instead of individual fruits is key to deal with non-rigidity, occlusions and challenging imagery with few distinct visual features to track. The results show that the proposed method can be successfully used to match fruits across videos and through time, and also to build an orchard map and later use it to locate the camera pose in 6DoF, thus providing a method for autonomous navigation of robots in the orchard and for selective fruit picking, for example.

翻译：通过植被周期追踪作物生长使农民能够在早期阶段预测坐果与产量，但若由人工使用卡尺或树木测量仪手动测量果实尺寸，则是一项繁重且不可扩展的任务。近年来，计算机视觉已被用于自动化精准农业中的多项任务，例如果实检测与计数以及尺寸估算。然而，将特定日期采集视频中的确切果实与后续日期采集视频中可见果实进行匹配的核心问题——即追踪果实随时间生长的必要步骤——仍有待解决。已有少量尝试，但它们要么假设相机始终从同一已知位置开始且存在足够独特的特征用于匹配，要么依赖GPS等其他数据源。本文提出了一种基于三维质心星座的新范式来解决该问题，并引入了一种适用于极稀疏三维点云的描述符，可用于跨视频匹配果实。匹配星座而非单个果实是处理非刚性、遮挡以及视觉特征稀少难以追踪的挑战性图像的关键。结果表明，所提方法能成功用于跨视频及时序的果实匹配，并能构建果园地图进而用于六自由度相机姿态定位，从而为果园机器人自主导航及选择性果实采摘（例如）提供方法。