Online video streaming has fundamental limitations on the transmission bandwidth and computational capacity and super-resolution is a promising potential solution. However, applying existing video super-resolution methods to online streaming is non-trivial. Existing video codecs and streaming protocols (\eg, WebRTC) dynamically change the video quality both spatially and temporally, which leads to diverse and dynamic degradations. Furthermore, online streaming has a strict requirement for latency that most existing methods are less applicable. As a result, this paper focuses on the rarely exploited problem setting of online streaming video super resolution. To facilitate the research on this problem, a new benchmark dataset named LDV-WebRTC is constructed based on a real-world online streaming system. Leveraging the new benchmark dataset, we proposed a novel method specifically for online video streaming, which contains a convolution and Look-Up Table (LUT) hybrid model to achieve better performance-latency trade-off. To tackle the changing degradations, we propose a mixture-of-expert-LUT module, where a set of LUT specialized in different degradations are built and adaptively combined to handle different degradations. Experiments show our method achieves 720P video SR around 100 FPS, while significantly outperforms existing LUT-based methods and offers competitive performance compared to efficient CNN-based methods.

翻译：在线视频流在传输带宽和计算能力方面存在根本性限制，超分辨率是一种有潜力的解决方案。然而，将现有视频超分辨率方法应用于在线流媒体并非易事。现有视频编解码器和流协议（如 WebRTC）会在空间和时间上动态改变视频质量，导致多样且动态的退化。此外，在线流媒体对延迟有严格要求，大多数现有方法难以适用。因此，本文聚焦于一个鲜少被探索的问题设定——在线流视频超分辨率。为促进该问题的研究，我们基于真实世界在线流系统构建了名为 LDV-WebRTC 的新型基准数据集。利用该基准数据集，我们提出了一种专用于在线视频流的新方法，该方法包含卷积与查找表（LUT）混合模型，以实现更优的性能-延迟权衡。为应对不断变化的退化情况，我们提出了混合专家LUT模块，其中构建了一组专用于不同退化场景的LUT，并通过自适应组合来处理各种退化。实验表明，我们的方法能在大约100 FPS下实现720P视频超分辨率，显著优于现有基于LUT的方法，同时在与高效CNN方法相比时展现出具有竞争力的性能。