With the rapid growth of multimedia data volume, there is an increasing need for efficient video transmission in applications such as virtual reality and future video streaming services. Semantic communication is emerging as a vital technique for ensuring efficient and reliable transmission in low-bandwidth, high-noise settings. However, most current approaches focus on joint source-channel coding (JSCC) that depends on end-to-end training. These methods often lack an interpretable semantic representation and struggle with adaptability to various downstream tasks. In this paper, we introduce the use of object-attribute-relation (OAR) as a semantic framework for videos to facilitate low bit-rate coding and enhance the JSCC process for more effective video transmission. We utilize OAR sequences for both low bit-rate representation and generative video reconstruction. Additionally, we incorporate OAR into the image JSCC model to prioritize communication resources for areas more critical to downstream tasks. Our experiments on traffic surveillance video datasets assess the effectiveness of our approach in terms of video transmission performance. The empirical findings demonstrate that our OAR-based video coding method not only outperforms H.265 coding at lower bit-rates but also synergizes with JSCC to deliver robust and efficient video transmission.
翻译:随着多媒体数据量的快速增长,虚拟现实和未来视频流服务等应用对高效视频传输的需求日益增长。语义通信正成为在低带宽、高噪声环境下确保高效可靠传输的关键技术。然而,当前大多数方法侧重于依赖端到端训练的联合信源信道编码(JSCC)。这些方法通常缺乏可解释的语义表示,并且难以适应各种下游任务。本文引入对象-属性-关系(OAR)作为视频的语义框架,以促进低比特率编码并增强JSCC过程,从而实现更有效的视频传输。我们利用OAR序列进行低比特率表示和生成式视频重建。此外,我们将OAR整合到图像JSCC模型中,以优先分配通信资源给对下游任务更关键的区域。我们在交通监控视频数据集上的实验评估了所提方法在视频传输性能方面的有效性。实证结果表明,我们基于OAR的视频编码方法不仅在较低比特率下优于H.265编码,而且能与JSCC协同工作,提供鲁棒且高效的视频传输。