Video frame interpolation (VFI) is the task that synthesizes the intermediate frame given two consecutive frames. Most of the previous studies have focused on appropriate frame warping operations and refinement modules for the warped frames. These studies have been conducted on natural videos containing only continuous motions. However, many practical videos contain various unnatural objects with discontinuous motions such as logos, user interfaces and subtitles. We propose three techniques to make the existing deep learning-based VFI architectures robust to these elements. First is a novel data augmentation strategy called figure-text mixing (FTM) which can make the models learn discontinuous motions during training stage without any extra dataset. Second, we propose a simple but effective module that predicts a map called discontinuity map (D-map), which densely distinguishes between areas of continuous and discontinuous motions. Lastly, we propose loss functions to give supervisions of the discontinuous motion areas which can be applied along with FTM and D-map. We additionally collect a special test benchmark called Graphical Discontinuous Motion (GDM) dataset consisting of some mobile games and chatting videos. Applied to the various state-of-the-art VFI networks, our method significantly improves the interpolation qualities on the videos from not only GDM dataset, but also the existing benchmarks containing only continuous motions such as Vimeo90K, UCF101, and DAVIS.

翻译：视频帧插值（VFI）是一项利用两个连续帧合成中间帧的任务。以往研究大多关注合适的帧扭曲操作及扭曲帧的优化模块，这些研究均基于仅包含连续运动的自然视频。然而，实际视频中常包含各类具有不连续运动的非自然物体，例如标志、用户界面及字幕。我们提出三种技术，使现有基于深度学习的VFI架构对这些元素具有鲁棒性。首先，提出一种名为"图文混合"(FTM)的新型数据增强策略，该策略无需额外数据集即可使模型在训练阶段学习不连续运动。其次，设计一个简洁有效的模块用于预测名为"不连续映射图"(D-map)的映射，该映射能密集区分连续运动与不连续运动区域。最后，提出可配合FTM与D-map使用的损失函数，用于监督不连续运动区域。我们额外构建了名为"图形化不连续运动"(GDM)的专用测试基准数据集，其中包含部分手机游戏及聊天视频。将我们的方法应用于多种最先进的VFI网络后，不仅能在GDM数据集上，还能在仅含连续运动的现有基准（如Vimeo90K、UCF101、DAVIS）上显著提升视频插值质量。