Temporal convolutional networks (TCNs) are a commonly used architecture for temporal video segmentation. TCNs however, tend to suffer from over-segmentation errors and require additional refinement modules to ensure smoothness and temporal coherency. In this work, we propose a novel temporal encoder-decoder to tackle the problem of sequence fragmentation. In particular, the decoder follows a coarse-to-fine structure with an implicit ensemble of multiple temporal resolutions. The ensembling produces smoother segmentations that are more accurate and better-calibrated, bypassing the need for additional refinement modules. In addition, we enhance our training with a multi-resolution feature-augmentation strategy to promote robustness to varying temporal resolutions. Finally, to support our architecture and encourage further sequence coherency, we propose an action loss that penalizes misclassifications at the video level. Experiments show that our stand-alone architecture, together with our novel feature-augmentation strategy and new loss, outperforms the state-of-the-art on three temporal video segmentation benchmarks.

翻译：时间变迁网络(TCNs)是一个常用的时间视频分割结构。但是,TCNs往往会遇到过度分解错误,需要额外的精细模块,以确保平稳和时间的一致性。在这项工作中,我们提议了一个新的时间编码解码器,以解决序列分解问题。特别是,解码器遵循粗到链结构,隐含着多个时间分辨率的组合。组合产生更准确和更好的分解,绕过额外精细化模块的需要。此外,我们用一个多分辨率特征强化战略加强我们的培训,以促进对不同时间分解的稳健性。最后,为了支持我们的架构并鼓励进一步的顺序一致性,我们建议了一种行动损失,以惩罚视频层次的分类错误。实验表明,我们的独立结构,加上我们新的地貌分解战略和新的损失,超越了三个时间视频分解基准的状态。