Video object segmentation approaches primarily rely on large-scale pixel-accurate human-annotated datasets for model development. In Dense Video Object Segmentation (DVOS) scenarios, each video frame encompasses hundreds of small, dense, and partially occluded objects. Accordingly, the labor-intensive manual annotation of even a single frame often takes hours, which hinders the development of DVOS for many applications. Furthermore, in videos with dense patterns, following a large number of objects that move in different directions poses additional challenges. To address these challenges, we proposed a semi-self-supervised spatiotemporal approach for DVOS utilizing a diffusion-based method through multi-task learning. Emulating real videos' optical flow and simulating their motion, we developed a methodology to synthesize computationally annotated videos that can be used for training DVOS models; The model performance was further improved by utilizing weakly labeled (computationally generated but imprecise) data. To demonstrate the utility and efficacy of the proposed approach, we developed DVOS models for wheat head segmentation of handheld and drone-captured videos, capturing wheat crops in fields of different locations across various growth stages, spanning from heading to maturity. Despite using only a few manually annotated video frames, the proposed approach yielded high-performing models, achieving a Dice score of 0.82 when tested on a drone-captured external test set. While we showed the efficacy of the proposed approach for wheat head segmentation, its application can be extended to other crops or DVOS in other domains, such as crowd analysis or microscopic image analysis.

翻译：视频目标分割方法主要依赖大规模像素级人工标注数据集进行模型开发。在密集视频目标分割场景中，每帧视频包含数百个小型、密集且部分遮挡的目标。因此，即使单帧图像的人工标注也常需数小时，这阻碍了DVOS在许多应用中的发展。此外，在具有密集模式的视频中，追踪大量沿不同方向运动的目标带来了额外挑战。为应对这些挑战，我们提出了一种基于扩散方法的半自监督时空DVOS方法，通过多任务学习实现。通过模拟真实视频的光流并生成其运动模式，我们开发了合成计算标注视频的方法论，可用于训练DVOS模型；利用弱标注（计算生成但不精确）数据进一步提升了模型性能。为验证所提方法的实用性与有效性，我们开发了针对手持设备和无人机拍摄视频的麦穗分割DVOS模型，这些视频采集了不同地区、从抽穗期到成熟期多个生长阶段的小麦作物。尽管仅使用少量人工标注视频帧，所提方法仍能生成高性能模型，在无人机拍摄的外部测试集上达到0.82的Dice分数。虽然我们展示了该方法在麦穗分割中的有效性，但其应用可扩展至其他作物或其他领域的DVOS任务，如人群分析或显微图像分析。