The quantitative analysis of microscope videos often requires instance segmentation and tracking of cellular and subcellular objects. The traditional method is composed of two stages: (1) performing instance object segmentation of each frame, and (2) associating objects frame-by-frame. Recently, pixel-embedding-based deep learning approaches provide single stage holistic solutions to tackle instance segmentation and tracking simultaneously. However, such deep learning methods require consistent annotations not only spatially (for segmentation), but also temporally (for tracking). In computer vision, annotated training data with consistent segmentation and tracking is resource intensive, the severity of which can be multiplied in microscopy imaging due to (1) dense objects (e.g., overlapping or touching), and (2) high dynamics (e.g., irregular motion and mitosis). To alleviate the lack of such annotations in dynamics scenes, adversarial simulations have provided successful solutions in computer vision, such as using simulated environments (e.g., computer games) to train real-world self-driving systems. In this paper, we propose an annotation-free synthetic instance segmentation and tracking (ASIST) method with adversarial simulation and single-stage pixel-embedding based learning. The contribution of this paper is three-fold: (1) the proposed method aggregates adversarial simulations and single-stage pixel-embedding based deep learning; (2) the method is assessed with both the cellular (i.e., HeLa cells) and subcellular (i.e., microvilli) objects; and (3) to the best of our knowledge, this is the first study to explore annotation-free instance segmentation and tracking study for microscope videos. This ASIST method achieved an important step forward, when compared with fully supervised approaches.

翻译：对显微镜视频的定量分析往往要求对细胞和子细胞对象进行分解和跟踪。传统方法由两个阶段组成:(1) 执行每个框架的试物分解,和(2) 将对象逐个框架联系起来。最近,以像素为基的深层学习方法提供了单一阶段的整体解决方案,以同时处理体分解和跟踪。然而,这种深层次的学习方法不仅需要空间(分解),而且需要时间(跟踪)。在计算机愿景中,具有一致分解和跟踪功能的附加说明的培训数据是资源密集型的,其严重程度可以乘以显微镜成像,因为:(1) 密集的物体(例如,重叠或触摸),以及(2) 高动态(例如,不规则的动作和分裂的深层学习方法),为了缓解动态场景中缺少这种说明的情况,对立模拟的模拟方法为模拟环境(例如,计算机游戏)培训真实世界的自我驱动系统。在本文中,我们建议采用无注释的合成实例分解和跟踪(ASIST) 方法,这是基于对立式的模拟和单阶段的模拟方法的模拟学习方法,这是基于模拟和单阶段的模拟方法。