Traditional video compression algorithms exhibit significant quality degradation at extremely low bitrates. Promptus emerges as a new paradigm for video streaming, substantially cutting down the bandwidth essential for video streaming. However, Promptus is computationally intensive and can not run in real-time on mobile devices. This paper presents PromptMobile, an efficient acceleration framework tailored for on-device Promptus. Specifically, we propose (1) a two-stage efficient generation framework to reduce computational cost by 8.1x, (2) a fine-grained inter-frame caching to reduce redundant computations by 16.6\%, (3) system-level optimizations to further enhance efficiency. The evaluations demonstrate that compared with the original Promptus, PromptMobile achieves a 13.6x increase in image generation speed. Compared with other streaming methods, PromptMobile achives an average LPIPS improvement of 0.016 (compared with H.265), reducing 60\% of severely distorted frames (compared to VQGAN).
翻译:传统视频压缩算法在极低比特率下表现出显著的质量下降。Promptus作为一种新的视频流传输范式应运而生,大幅削减了视频流传输所需的带宽。然而,Promptus计算密集,无法在移动设备上实时运行。本文提出了PromptMobile,一个专为设备端Promptus设计的高效加速框架。具体而言,我们提出了(1)一个两阶段高效生成框架,将计算成本降低8.1倍;(2)一种细粒度帧间缓存机制,将冗余计算减少16.6%;(3)系统级优化以进一步提升效率。评估结果表明,与原始Promptus相比,PromptMobile实现了图像生成速度13.6倍的提升。与其他流传输方法相比,PromptMobile实现了平均LPIPS指标0.016的改进(相较于H.265),并减少了60%的严重失真帧(相较于VQGAN)。