Modern displays are capable of rendering video content with high dynamic range (HDR) and wide color gamut (WCG). However, the majority of available resources are still in standard dynamic range (SDR). As a result, there is significant value in transforming existing SDR content into the HDRTV standard. In this paper, we define and analyze the SDRTV-to-HDRTV task by modeling the formation of SDRTV/HDRTV content. Our analysis and observations indicate that a naive end-to-end supervised training pipeline suffers from severe gamut transition errors. To address this issue, we propose a novel three-step solution pipeline called HDRTVNet++, which includes adaptive global color mapping, local enhancement, and highlight refinement. The adaptive global color mapping step uses global statistics as guidance to perform image-adaptive color mapping. A local enhancement network is then deployed to enhance local details. Finally, we combine the two sub-networks above as a generator and achieve highlight consistency through GAN-based joint training. Our method is primarily designed for ultra-high-definition TV content and is therefore effective and lightweight for processing 4K resolution images. We also construct a dataset using HDR videos in the HDR10 standard, named HDRTV1K that contains 1235 and 117 training images and 117 testing images, all in 4K resolution. Besides, we select five metrics to evaluate the results of SDRTV-to-HDRTV algorithms. Our final results demonstrate state-of-the-art performance both quantitatively and visually. The code, model and dataset are available at https://github.com/xiaom233/HDRTVNet-plus.
翻译:现代显示器能够呈现高动态范围(HDR)和宽色域(WCG)的视频内容。然而,现有资源大多仍为标准动态范围(SDR)。因此,将现有SDR内容转换为HDRTV标准具有重要价值。本文通过建模SDRTV/HDRTV内容的形成过程,定义并分析了SDRTV-to-HDRTV任务。我们的分析与观察表明,朴素的端到端监督训练流程存在严重的色域转换错误。为解决这一问题,我们提出了一种新型三步解决方案流程HDRTVNet++,包括自适应全局颜色映射、局部增强和高光优化。自适应全局颜色映射步骤以全局统计信息为引导,执行图像自适应的颜色映射;随后部署局部增强网络以提升细节;最后,我们将上述两个子网络组合为生成器,并通过基于GAN的联合训练实现高光一致性。该方法主要面向超高清电视内容设计,因此能够高效且轻量地处理4K分辨率图像。我们还利用HDR10标准的HDR视频构建了名为HDRTV1K的数据集,包含1235张训练图像和117张测试图像,均为4K分辨率。此外,我们选取了五个评估指标来评价SDRTV-to-HDRTV算法的结果。最终结果在定量指标和视觉质量上均达到了最先进水平。代码、模型和数据集已开源至https://github.com/xiaom233/HDRTVNet-plus。