Reshaping, a point operation that alters the characteristics of signals, has been shown capable of improving the compression ratio in video coding practices. Out-of-loop reshaping that directly modifies the input video signal was first adopted as the supplemental enhancement information (SEI) for the HEVC/H.265 without the need to alter the core design of the video codec. VVC/H.266 further improves the coding efficiency by adopting in-loop reshaping that modifies the residual signal being processed in the hybrid coding loop. In this paper, we theoretically analyze the rate-distortion performance of the in-loop reshaping and use experiments to verify the theoretical result. We prove that the in-loop reshaping can improve coding efficiency when the entropy coder adopted in the coding pipeline is suboptimal, which is in line with the practical scenarios that video codecs operate in. We derive the PSNR gain in a closed form and show that the theoretically predicted gain is consistent with that measured from experiments using standard testing video sequences.

翻译：整形作为一种改变信号特性的点操作，已被证明能够提升视频编码实践中的压缩率。直接修改输入视频信号的环路外整形首先作为补充增强信息（SEI）被HEVC/H.265标准采纳，无需改动视频编解码器的核心设计。VVC/H.266则通过采用环路内整形进一步提升了编码效率，该技术可修改混合编码环路中正在处理的残差信号。本文从理论上分析了环路内整形的率失真性能，并通过实验验证了理论结果。我们证明当编码流水线中采用的熵编码器处于次优状态时（这与视频编解码器的实际运行场景相符），环路内整形能够提升编码效率。我们以闭合形式推导了PSNR增益，并表明理论预测的增益与采用标准测试视频序列的实验测量结果一致。