Food instance segmentation is essential to estimate the serving size of dishes in a food image. The recent cutting-edge techniques for instance segmentation are deep learning networks with impressive segmentation quality and fast computation. Nonetheless, they are hungry for data and expensive for annotation. This paper proposes an incremental learning framework to optimize the model performance given a limited data labelling budget. The power of the framework is a novel difficulty assessment model, which forecasts how challenging an unlabelled sample is to the latest trained instance segmentation model. The data collection procedure is divided into several stages, each in which a new sample package is collected. The framework allocates the labelling budget to the most difficult samples. The unlabelled samples that meet a certain qualification from the assessment model are used to generate pseudo-labels. Eventually, the manual labels and pseudo-labels are sent to the training data to improve the instance segmentation model. On four large-scale food datasets, our proposed framework outperforms current incremental learning benchmarks and achieves competitive performance with the model trained on fully annotated samples.

翻译：食物实例分割对于从食物图像中估算菜肴份量至关重要。当前最先进的实例分割技术采用深度学习网络，在分割质量与计算效率上表现出色。然而，这些方法对数据需求量大、标注成本高昂。本文提出一种增量学习框架，旨在有限的数据标注预算下优化模型性能。该框架的核心是一种新颖的难度评估模型，能够预测未标注样本对当前训练的实例分割模型而言的挑战程度。数据收集过程分为多个阶段，每个阶段收集一组新样本。该框架将标注预算分配给最困难的样本，并利用难度评估模型中符合特定条件的未标注样本生成伪标签。最终，人工标注与伪标签共同加入训练数据以优化实例分割模型。在四个大规模食物数据集上的实验表明，所提框架优于现有增量学习基准方法，并在性能上与使用全标注样本训练的模型相当。