Online video streaming has fundamental limitations on the transmission bandwidth and computational capacity and super-resolution is a promising potential solution. However, applying existing video super-resolution methods to online streaming is non-trivial. Existing video codecs and streaming protocols (\eg, WebRTC) dynamically change the video quality both spatially and temporally, which leads to diverse and dynamic degradations. Furthermore, online streaming has a strict requirement for latency that most existing methods are less applicable. As a result, this paper focuses on the rarely exploited problem setting of online streaming video super resolution. To facilitate the research on this problem, a new benchmark dataset named LDV-WebRTC is constructed based on a real-world online streaming system. Leveraging the new benchmark dataset, we proposed a novel method specifically for online video streaming, which contains a convolution and Look-Up Table (LUT) hybrid model to achieve better performance-latency trade-off. To tackle the changing degradations, we propose a mixture-of-expert-LUT module, where a set of LUT specialized in different degradations are built and adaptively combined to handle different degradations. Experiments show our method achieves 720P video SR around 100 FPS, while significantly outperforms existing LUT-based methods and offers competitive performance compared to efficient CNN-based methods.

翻译：在线视频流媒体在传输带宽和计算能力上存在根本性限制，超分辨率技术是一种具有潜力的解决方案。然而，将现有的视频超分辨率方法直接应用于在线流媒体并非易事。现有的视频编解码器和流媒体协议（例如WebRTC）会在空间和时间上动态改变视频质量，导致多样且动态的退化。此外，在线流媒体对延迟有严格的要求，而大多数现有方法难以满足这一条件。因此，本文聚焦于一个鲜有探索的问题设置——在线流媒体视频超分辨率。为促进该问题的研究，我们基于真实的在线流媒体系统构建了一个名为LDV-WebRTC的新型基准数据集。借助该基准数据集，我们提出了一种专用于在线视频流媒体的新方法，该方法包含一个卷积与查找表（LUT）的混合模型，以实现更好的性能-延迟权衡。为应对变化的退化，我们提出了一种专家混合LUT模块，构建了一组专用于不同退化的LUT，并通过自适应组合来处理不同的退化情况。实验表明，我们的方法能以约100 FPS的速度实现720P视频超分辨率，同时显著优于现有的基于LUT的方法，并在性能上与高效的基于CNN的方法相媲美。