Assigning consistent temporal identifiers to multiple moving objects in a video sequence is a challenging problem. A solution to that problem would have immediate ramifications in multiple object tracking and segmentation problems. We propose a strategy that treats the temporal identification task as a spatio-temporal clustering problem. We propose an unsupervised learning approach using a convolutional and fully connected autoencoder, which we call deep heterogeneous autoencoder, to learn discriminative features from segmentation masks and detection bounding boxes. We extract masks and their corresponding bounding boxes from a pretrained instance segmentation network and train the autoencoders jointly using task-dependent uncertainty weights to generate common latent features. We then construct constraints graphs that encourage associations among objects that satisfy a set of known temporal conditions. The feature vectors and the constraints graphs are then provided to the kmeans clustering algorithm to separate the corresponding data points in the latent space. We evaluate the performance of our method using challenging synthetic and real-world multiple-object video datasets. Our results show that our technique outperforms several state-of-the-art methods.

翻译：在视频序列中向多个移动对象指派一致的时间识别器是一个具有挑战性的问题。这个问题的解决方案将在多个对象的跟踪和分割问题中产生直接的影响。我们提出了一个战略,将时间识别任务作为时空聚变问题处理。我们建议采用一个不受监督的学习方法,使用一个连接和完全连接的自动编码器,我们称之为深异质自动编码器,从隔段遮罩和检测捆绑盒中学习歧视特征。我们从一个预先培训的试样分割网中提取面具及其对应的捆绑框,并用依赖任务的不确定性重量联合培训自动编码器以产生共同的潜伏特征。我们然后构建制约图,鼓励满足一组已知时间条件的物体之间的关联。然后向千米人组合算法提供特性矢量和制约图,以分离潜空的相应数据点。我们用挑战合成和真实世界多位截图视频数据集来评估我们方法的性能。我们的结果显示,我们的技术优于几种状态的方法。