Deep learning-based alpha matting showed tremendous improvements in recent years, yet, feature film production studios still rely on classical chroma keying including costly post-production steps. This perceived discrepancy can be explained by some missing links necessary for production which are currently not adequately addressed in the alpha matting community, in particular foreground color estimation or color spill compensation. We propose a neural network-based temporal multi-backdrop production system that combines beneficial features from chroma keying and alpha matting. Given two consecutive frames with different background colors, our one-encoder-dual-decoder network predicts foreground colors and alpha values using a patch-based overlap-blend approach. The system is able to handle imprecise backdrops, dynamic cameras, and dynamic foregrounds and has no restrictions on foreground colors. We compare our method to state-of-the-art algorithms using benchmark datasets and a video sequence captured by a demonstrator setup. We verify that a dual backdrop input is superior to the usually applied trimap-based approach. In addition, the proposed studio set is actor friendly, and produces high-quality, temporal consistent alpha and color estimations that include a superior color spill compensation.

翻译：基于深度学习的alpha抠图技术近年来取得了显著进展，但电影长片制作工作室仍依赖包括昂贵后期处理步骤的传统色键抠像技术。这种认知差异源于现有alpha抠图研究领域尚未充分解决影视制作所需的若干关键环节，特别是前景色彩估计与色彩溢出补偿问题。我们提出一种基于神经网络的时序多背景制作系统，融合了色键抠像与alpha抠图的优势功能。通过采用不同背景色的连续两帧图像，我们的单编码器-双解码器网络利用基于补丁的重叠混合方法预测前景色彩与alpha值。该系统能够处理非精确背景、动态摄像机及动态前景场景，且对前景色彩无任何限制。我们使用基准数据集和演示装置采集的视频序列，将本方法与现有最优算法进行对比，验证了双背景输入方案相较于传统三分图方法的优越性。此外，所提出的演播室配置具有演员友好特性，可生成高质量、时序一致的alpha值与色彩估计结果，并具备卓越的色彩溢出补偿能力。