The rate-distortion curve captures the fundamental tradeoff between compression length and resolution in lossy data compression. However, it conceals the underlying dynamics of optimal source encodings or test channels. We argue that these typically follow a piecewise smooth trajectory as the source information is compressed. These smooth dynamics are interrupted at bifurcations, where solutions change qualitatively. Sub-optimal test channels may collide or exchange optimality there, for example. There is typically a plethora of sub-optimal solutions, which stems from restrictions of the reproduction alphabet. We devise a family of algorithms that exploits the underlying dynamics to track a given test channel along the rate-distortion curve. To that end, we express implicit derivatives at the roots of a non-linear operator by higher derivative tensors. Providing closed-form formulae for the derivative tensors of Blahut's algorithm thus yields implicit derivatives of arbitrary order at a given test channel, thereby approximating others in its vicinity. Finally, our understanding of bifurcations guarantees the optimality of the root being traced, under mild assumptions, while allowing us to detect when our assumptions fail. Beyond the interest in rate distortion, this is an example of how understanding a problem's bifurcations can be translated to a numerical algorithm.
翻译:率失真曲线刻画了有损数据压缩中压缩长度与分辨率之间的基本权衡关系。然而,该曲线掩盖了最优信源编码或测试信道的潜在动力学特性。我们论证,当信源信息被压缩时,这些特性通常遵循分段光滑的轨迹。这些光滑动力学在分岔点处中断,此时解发生质的改变。例如,次优测试信道可能在该处发生碰撞或交换最优性。通常,大量次优解的存在源于重构字母集的限制。我们设计了一系列算法,利用潜在动力学特性沿率失真曲线追踪给定测试信道。为此,我们在非线性算子根处通过高阶导数张量表达隐式导数。通过提供Blahut算法导数张量的闭式公式,可在给定测试信道处获得任意阶隐式导数,进而近似其邻域内的其他解。最后,基于对分岔现象的理解,在温和假设下保证了所追踪根的最优性,同时使我们能够检测假设失效的情况。除率失真领域的兴趣外,本文还展示了如何将问题的分岔分析转化为数值算法的一个范例。