The paper introduces the application of information geometry to describe the ground states of Ising models by utilizing parity-check matrices of cyclic and quasi-cyclic codes on toric and spherical topologies. The approach establishes a connection between machine learning and error-correcting coding. This proposed approach has implications for the development of new embedding methods based on trapping sets. Statistical physics and number geometry applied for optimize error-correcting codes, leading to these embedding and sparse factorization methods. The paper establishes a direct connection between DNN architecture and error-correcting coding by demonstrating how state-of-the-art architectures (ChordMixer, Mega, Mega-chunk, CDIL, ...) from the long-range arena can be equivalent to of block and convolutional LDPC codes (Cage-graph, Repeat Accumulate). QC codes correspond to certain types of chemical elements, with the carbon element being represented by the mixed automorphism Shu-Lin-Fossorier QC-LDPC code. The connections between Belief Propagation and the Permanent, Bethe-Permanent, Nishimori Temperature, and Bethe-Hessian Matrix are elaborated upon in detail. The Quantum Approximate Optimization Algorithm (QAOA) used in the Sherrington-Kirkpatrick Ising model can be seen as analogous to the back-propagation loss function landscape in training DNNs. This similarity creates a comparable problem with TS pseudo-codeword, resembling the belief propagation method. Additionally, the layer depth in QAOA correlates to the number of decoding belief propagation iterations in the Wiberg decoding tree. Overall, this work has the potential to advance multiple fields, from Information Theory, DNN architecture design (sparse and structured prior graph topology), efficient hardware design for Quantum and Classical DPU/TPU (graph, quantize and shift register architect.) to Materials Science and beyond.
翻译:本文介绍利用信息几何描述伊辛模型基态的方法,通过采用环面与球面拓扑上循环码和准循环码的校验矩阵,建立机器学习与纠错编码之间的关联。该方案对基于陷阱集的嵌入方法发展具有启示意义。统计物理与数论几何被应用于优化纠错编码,进而衍生出嵌入与稀疏分解方法。论文通过论证长程任务中的前沿架构(ChordMixer、Mega、Mega-chunk、CDIL等)可等效于分组卷积LDPC码(Cage图、重复累积码),直接建立了深度神经网络架构与纠错编码的联系。准循环码对应特定化学元素类型,其中碳元素由混合自同构Shu-Lin-Fossorier QC-LDPC码表征。本文详细阐述了置信传播与积和式、Bethe积和式、西森温度及Bethe-海森矩阵之间的关联。用于Sherrington-Kirkpatrick伊辛模型的量子近似优化算法,可视为深度神经网络训练中反向传播损失函数景观的模拟。这种相似性产生了与TS伪码字类似的问题,其形态类似于置信传播方法。此外,QAOA中的层深度与Wiberg译码树中的置信传播译码迭代次数存在对应关系。总体而言,这项研究有望推动信息论、深度神经网络架构设计(稀疏与结构化先验图拓扑)、量子与经典DPU/TPU(图处理、量化与移位寄存器架构)高效硬件设计、材料科学等多个领域的发展。