Most computer vision research focuses on datasets containing thousands of images of commonplace objects. However, many high-impact datasets, such as those in medicine and the geosciences, contain fine-grain objects that require domain-expert knowledge to recognize and are time-consuming to collect and annotate. As a result, these datasets contain few labeled images, and current machine vision models cannot train intensively on them. Originally introduced to correct large-language models, model-editing techniques in machine learning have been shown to improve model performance using only small amounts of data and additional training. Using a Mask R-CNN to segment ancient reef fossils in rock sample images, we present a two-part paradigm to improve fossil segmentation with few labeled images: we first identify model weaknesses using image perturbations and then mitigate those weaknesses using model editing. Specifically, we apply domain-informed image perturbations to expose the Mask R-CNN's inability to distinguish between different classes of fossils and its inconsistency in segmenting fossils with different textures. To address these shortcomings, we extend an existing model-editing method for correcting systematic mistakes in image classification to image segmentation with no additional labeled data needed and show its effectiveness in decreasing confusion between different kinds of fossils. We also highlight the best settings for model editing in our situation: making a single edit using all relevant pixels in one image (vs. using multiple images, multiple edits, or fewer pixels). Though we focus on fossil segmentation, our approach may be useful in other similar fine-grain segmentation problems where data is limited.

翻译：大多数计算机视觉研究关注的是包含数千张常见物体图像的数据集。然而，许多高影响力数据集（如医学和地球科学领域）包含需要领域专家知识才能识别的细粒度物体，且收集与标注耗时巨大。因此，此类数据集仅有少量标注图像，现有机器视觉模型难以在其上进行密集训练。模型编辑技术最初用于纠正大语言模型，近期研究表明，仅需少量数据和额外训练即可提升模型性能。我们利用Mask R-CNN分割岩石样本图像中的远古礁石化石，提出了一种双阶段范式，通过少量标注图像改进化石分割：首先利用图像扰动识别模型缺陷，随后通过模型编辑消除这些缺陷。具体而言，我们应用领域引导的图像扰动，揭示Mask R-CNN在区分不同化石类别时的不足，以及其在不同纹理化石分割中的不一致性。为解决上述问题，我们将现有针对图像分类系统性错误的模型编辑方法扩展至图像分割任务，无需额外标注数据，并证明该方法能有效降低不同化石类别间的混淆度。此外，我们明确了最优编辑设置：使用单张图像中所有相关像素进行单次编辑（而非多张图像、多次编辑或更少像素）。尽管本文聚焦化石分割，但该方案对数据受限的其他细粒度分割问题同样具有参考价值。